Google releases gemini-decision-1: a model that returns probabilities instead of text in 150–250 ms

A familiar scene: you need to decide which queue a customer request should go to. You take a large language model, ask it to "answer in JSON format," wait while it generates text token by token, then parse the response and pray that no bracket got lost. All of this for a single decision of the "branch A or branch B" kind. Google has released a tool that removes this layer entirely.

What exactly has been released

The model is called gemini-decision-1. It is available as a separate endpoint and is currently in preview in AI Studio. The main difference from regular Gemini is that it doesn't generate tokens at all. Instead, in a single forward pass it scores the answer options and returns probabilities. Output tokens are zero, and your code can branch directly on the numbers.

The input can be text, images, audio, video and PDF, up to 262,144 tokens in total. You can ask up to 1,000 independent questions about a single input. There are three question types:

• yes/no, with an answer in the range 0.01–0.99;
• a choice among 2–255 options;
• a rubric score with a weighted average.

The questions don't see each other, so the answers don't influence one another. According to the announcement, latency barely grows: 50–100 ms for the model itself and 150–250 ms end-to-end, the same for one question as for a hundred.

What it looks like in an example

Imagine an incoming support email with a screenshot attached. Previously you would make several calls, or one long prompt asking for a structured output. Now you send the email and screenshot once and add three questions:

• "Is this a payment problem?" — yes/no;
• "Which team should it go to?" — a choice among five options;
• "How urgent is it by the rubric?" — a score against criteria.

You get back three sets of numbers. After that it's a plain if: if the payment probability is above 0.8, send it to billing. No parsing and no retries because of a broken format.

The pricing model is also nice. You pay for the shared input once, and each question adds only its own instructions and criteria. There are no separate charges for thinking tokens, audio transcription or OCR. Price: $0.10 per 1M input tokens and $0 for output. The free tier offers 200 requests per minute, 50K tokens per minute and 10,000 requests per day. On Tier 1 the limits rise to 1,500 RPM and 1M TPM, on Tier 3 to 15,000 RPM and 10M TPM.

Who needs this and where the catch is

The author of the original breakdown calls this the most underrated type of release and bets on agents. If you have a ten-step chain and need to pick a tool at each step, then by his estimate the savings are not a matter of percentages but of an order of magnitude: less latency, less money, no parsing. This is the post author's estimate, not a measured benchmark, but the logic is clear. Many production LLM calls are essentially classifiers: routing, moderation, filters, triage, checklist scoring. For them, text generation was merely an adapter.

Now the limitations.

• This is a preview. Limits, prices and model behavior may change before release.
• Batch API and context caching are not supported. If you have a large repeated context, such as a long instruction or a knowledge base, you will have to pay for it every time.
• The description has no quality metrics. It is unknown how well the probabilities are calibrated and how the model handles subtle rubric wording. If you plan to make a decision with legal or financial consequences on a 0.7 threshold, check calibration on your own data first.
• This is a Google endpoint, so vendor lock-in remains.

What to do in practice

List your LLM calls and mark the ones where the answer ultimately boils down to a label, a flag or a number. These are the first candidates. Assemble a labeled sample of a few hundred examples, run it through the free tier in AI Studio, and compare accuracy and latency with your current solution. If quality is no worse and latency drops to 150–250 ms, routing and moderation are worth moving here. Text generation for the user, of course, should stay with regular models.