LLM 0.37 Adds Multimodal Embeddings and Merges Fixes from 12 Contributors

CLI tools for working with models quickly run into the same problem: every provider requires separate setup, and data long ago stopped being limited to text. In LLM 0.37, Simon Willison tackles both. The release simplifies connecting OpenAI-compatible APIs and lets you pass objects of different types into the embeddings system: text, audio, images and video.

In practice, this means less special-purpose code around models. If a team stores call transcripts, product photos and training videos, it no longer has to pretend in advance that all of it is the same type of content. Each object can get its own content type, and a suitable multimodal model can build a vector representation for it.

One interface for different providers

The new llm openai endpoint add command is intended for services that replicate the OpenAI API format. Previously, developers had to work out the configuration of an additional model by hand. Now connecting one is handled by a dedicated CLI command.

A simple scenario looks like this: a team has an internal gateway or a third-party provider with an OpenAI-compatible API. The developer adds its endpoint, specifies the available model, and then calls it through the familiar LLM interface. This is especially convenient for prototypes and internal tools, where models have to be swapped or compared regularly.

The release also adds support for gpt-6.1-sol and gpt-6.1-terra. The default model for users who have not chosen their own is now gpt-6-luna. According to Willison, it costs half as much as the previous default, gpt-5.6-luna. For existing installations with an explicitly set model, this change switches nothing.

Embeddings are no longer text-only

The main architectural change is support for different types of binary content when building embeddings. This is needed for models such as Gemini Embedding 2, which can work not only with text but also with audio, images and video.

Consider a search over a support knowledge base. A user types a text question, while the right answer may live in a manual, an interface screenshot, a conversation recording or a video clip. Multimodal embeddings make it possible to place such materials in a shared search space. LLM 0.37 does not build a ready-made search system itself, but it removes one annoying barrier: the library no longer requires treating all input objects as identical.

It is important here to check the capabilities of the chosen provider. Format support in the library does not mean that every connected model can process all four data types. Limits on file size, cost, formats and search quality also remain on the side of the specific model.

AI pull requests as detailed bug reports

Separately, Willison described his experience with AI-assisted contributions to an open source project. The release includes fixes from 12 contributors, and almost all of them were created with the help of AI. Instead of automatically trusting ready-made patches, the author used Opus 5.5 or GPT-6 Astra for an initial review of the proposals.

The useful idea here is not that AI code can be accepted without checking. Willison looks at such a pull request as a bug report accompanied by a reproducible example and a possible solution. Sometimes the patch fits; sometimes it is easier to fix the problem anew. This process lowers the cost of initial triage, but responsibility for the architecture and the final code still rests with the project maintainer.

Practical takeaway: teams that switch between providers or build search over mixed data should try LLM 0.37. And owners of open repositories can benefit from the editorial approach itself: separate the value of the problem found from the quality of the AI patch submitted, and evaluate them independently.