Kimi K3.1 Pro Works in vLLM from Day One: 1M Tokens and 65.7% on Terminal-Bench 4.0

A new open model usually follows a familiar path. First comes the release, then weeks of waiting for the community to finish the inference stack, while users run it through someone else's API. Moonshot AI's Kimi K3.1 Pro skipped that stage: the vLLM project announced support for the model on launch day.

What we know

Three facts stand out from the announcement. First, Kimi K3.1 Pro has a 1-million-token context window. Moonshot positions the model for long coding sessions, 3D content creation, and research tasks. Second, it scores 65.7% on Terminal-Bench 4.0. This benchmark requires an agent to complete tasks directly in a terminal: installing dependencies, fixing builds, and editing configuration files. The measurement used the Claude Code harness. Third, vLLM promises day-zero GPU serving, meaning the model can be launched immediately without waiting for forks and patches.

The post provides no further details. It does not specify the model's size, architecture, license, or memory requirements, so those will need to be checked in Moonshot's documentation and the vLLM release notes.

Why vLLM support matters

vLLM has long been one of the main engines for self-hosted inference. When a model works in it from day one, teams get a clear deployment path: take a ready-made image, expose an OpenAI-compatible endpoint, and connect it to existing tools. There is no need to write a custom server or deal with nonstandard kernels.

This matters for companies with strict data requirements. Customer code, internal repositories, and incident logs often cannot be sent to an external API. Previously, the choice was between a closed model accessed through an API and an open model that lagged behind. If the claimed results hold up, there is now a third option: a capable agentic model running on a company's own hardware.

What 1M tokens and 65.7% mean

One million tokens does not simply mean that “the entire codebase fits.” A long agent session accumulates history: command output, diffs, test logs, and previous attempts. A larger window reduces how often the context must be compressed and details discarded. But long context has a cost. The KV cache grows with sequence length and can quickly consume GPU memory under real workloads. “Supports 1M” and “comfortably handles 1M across dozens of concurrent sessions” are therefore very different claims, and only your own measurements will reveal the difference.

The 65.7% Terminal-Bench 4.0 score also requires context. Results depend on the harness, and this test used the Claude Code harness. A different agent framework, prompts, and toolset could produce a different figure. Moreover, the result was published by the developer itself. The post provides no independent reproductions.

What to do in practice

If you build coding agents or internal developer tools, a sensible plan is:

1. Read the vLLM release notes and check the hardware and configuration required for long context.
2. Select 20–30 real tasks from your backlog, such as fixing CI, performing migrations, and investigating failing tests. Run them through both Kimi K3.1 Pro and your current model.
3. Measure not only the task completion rate, but also GPU-hour cost, latency in long sessions, and the number of steps required to reach a result.
4. Test with your own agent harness, not only the one used to produce the benchmark.

Product teams should note another development: open models are reaching production infrastructure faster. The gap between releasing model weights and being able to use them has shrunk to zero. This reduces the risk of vendor lock-in and strengthens customers' position in pricing negotiations.

Bottom line: this is more an infrastructure story than a revolution. But if the figures are confirmed, teams with their own GPUs will gain a serious alternative to closed agentic models, without weeks of preparation.