Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableHabr AI→ original

vue-llm Library Brings an AI Chat to Vue Right in the Browser: Models From 400 MB

Ready-made tooling for AI chats is written mostly for React, so Vue and Nuxt developers have had to assemble response streaming and tool calls by hand. The vue-llm library offers a useChat composable that works with OpenAI, Anthropic, Gemini and Ollama. It also has a serverless mode: the model runs in the browser on the GPU via WebGPU. The first download is from 400 MB, and after that loading from cache takes a second or two. For Vue teams, it is a ready-made foundation instead of a homegrown one.

AI-processed from Habr AI; edited by Hamidun News
vue-llm Library Brings an AI Chat to Vue Right in the Browser: Models From 400 MB
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Key metric #1MIT license, version 1.3, Vue is the only runtime dependency
Key metric #2The API key stays on the server; the example sets a limit of 30 requests per minute
Key metric #3The response card fills in while the model is still writing the call

If you have built an AI feature on Vue (a popular frontend framework) or Nuxt (its extension for full-fledged websites), you have probably run into the same problem: almost all ready-made tooling for chats with language models is written for React. In Vue, you have to adapt it or hand-write streaming (showing the response as it is generated rather than all at once), tool calling and conversation state. A developer has released an open-source library, vue-llm, that handles this routine work for Vue 3, Nuxt and even Vue 2.

7. The most striking scenario is that a small model runs right in the user's browser, on their GPU, via WebGPU. There is no server and no API key, but there is a one-time model download of from 400 MB.

The library is distributed under the MIT license, is currently at version 1.3, and pulls in no runtime dependencies other than Vue itself.

A chat in ten lines and the key on the server

The core is the useChat composable. It streams the model's response token by token into reactive state: messages, status, error, cancellation, regeneration. Messages are made up of "parts": text, model reasoning, tool calls, files, sources. As a result, a single template renders an ordinary response, the reasoning of Claude or Qwen3, and a tool call.

The API key stays on the server. The server side is a plain web-standard Request → Response handler. It works in Nuxt/Nitro, Node, Bun, Deno and on edge runtimes. In the author's example, a rate limit of 30 requests per minute is enabled in the same place. For production, this is a useful detail: without it, a public chat quickly turns into free access for strangers to your token bill.

Out of the box, the library supports OpenAI and any OpenAI-compatible API, Anthropic, Google Gemini and Ollama. If you already have your own streaming backend, createSSEProvider connects it to useChat. You describe how your SSE events turn into chunks of the response, and everything else works as usual.

Generative UI and the human in the loop

The most interesting part is Generative UI. When the model calls a tool, you can show your own Vue component instead of text. Arguments are streamed into the component as they are generated: a weather card starts filling in while the model is still writing the call. To make this work, the author had to write a parser for "unfinished" JSON. The string {"city":"Mos becomes { city: 'Mos' }, and incomplete keys are dropped until they are finished.

A tool with an execute function runs on its own. A tool without one pauses the chat and waits for a response from the interface. This is how the human-in-the-loop pattern works, where a person makes the decision. For example, the model proposes booking a table, the user clicks "Confirm" in the card, addToolResult passes the answer back, and the conversation continues. For irreversible actions (payment, sending an email, deletion), this is more reliable than telling the model in the prompt to "ask for permission".

In-browser models and token economics

The createWebLLM provider runs open models (Llama 3.2, Qwen, SmolLM, Phi) on WebGPU via WebLLM. No server or key is needed, and data does not leave the device. In code, only the provider changes, and useChat stays the same as with OpenAI.

From a cost standpoint, this is an interesting move. Simple tasks such as rephrasing, classification or short autocompletion can be handed to a local model, so you do not pay for every token. The author also suggests a hybrid: give users without WebGPU a server-side model and everyone else a local one. The price of this approach is the first download of from 400 MB. After that, the model loads from the browser cache in a second or two. But compact models like Llama 3.2 1B in quantized form are noticeably weaker than cloud flagships, so complex reasoning and long context are better left to the server.

UI, DevTools and Nuxt

The library has a hybrid design. npm ships the headless logic (without ready-made styling): usePromptInput and useAutoScroll. They cover the annoying details: autoscroll that does not yank a user who is scrolling up; an Enter key that does not send the message while typing in Chinese or Japanese; pasting images from the clipboard and drag-and-drop of files. Ready-made components are copied into your project with the shadcn-vue command, after which the code belongs to you. They use shadcn theme tokens, so they adapt to the rest of your interface, including dark mode.

For debugging, there is the vue-llm/devtools module: in the LLM tab of Vue DevTools you can see every useChat, tokens, time to first token, generation speed, tool calls and a request timeline. Without the plugin, the library traces nothing and spends no resources on it. The Nuxt module installs with a single command, auto-imports composables and providers, and has been tested on Nuxt 3 and 4.

Limitations and what's next

The author compares the project with ready-made AI SDKs and calls it Vue-first. The differences are the Nuxt module, components for Generative UI, DevTools, a provider for in-browser models, an adapter for an existing SSE backend, and Vue 2.7 support for the composables, which matters for legacy admin panels. At the same time, the project is young and maintained by a single developer, so the maturity of its ecosystem is not yet comparable to React counterparts. Before adopting it, check how your providers and scenarios behave under real load.

The roadmap includes an MCP client for connecting MCP servers as tools, answers with citations for RAG, and voice input. For Vue teams that need a chat or an agent interface, the sandbox, which requires no API key, lets you evaluate the approach in a few minutes.

Story

Key Questions Answered

What are the key details on A chat in ten lines and the key on the server?

The core is the useChat composable. It streams the model's response token by token into reactive state: messages, status, error, cancellation, regeneration. Messages are made up of "parts": text, model reasoning, tool calls, files, sources. As a result, a single template renders an ordinary response, the reasoning of Claude or Qwen3, and a tool call. The API key stays on the server. The server side is a plain web-standard Request → Response handler. It works in Nuxt/Nitro, Node, Bun, Deno and on edge runtimes.

What are the key details on Generative UI and the human in the loop?

The most interesting part is Generative UI. When the model calls a tool, you can show your own Vue component instead of text. Arguments are streamed into the component as they are generated: a weather card starts filling in while the model is still writing the call. To make this work, the author had to write a parser for "unfinished" JSON. The string {"city":"Mos becomes { city: 'Mos' }, and incomplete keys are dropped until they are finished. A tool with an execute function runs on its own.

What are the key details on In-browser models and token economics?

The createWebLLM provider runs open models (Llama 3.2, Qwen, SmolLM, Phi) on WebGPU via WebLLM. No server or key is needed, and data does not leave the device. In code, only the provider changes, and useChat stays the same as with OpenAI. From a cost standpoint, this is an interesting move. Simple tasks such as rephrasing, classification or short autocompletion can be handed to a local model, so you do not pay for every token.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…