Google launches an API for choosing actions with 150–250 ms latency

A queue of support tickets has built up, and an LLM needs to decide where to send each one: billing, technical support or an account manager. To make a single choice, the team configures JSON generation, validates the format and handles failed responses. Google is offering a dedicated API for these tasks: the developer specifies the allowed options, and the model returns their probabilities. It is a promising candidate for experimentation wherever a product needs to choose its next action quickly.

In Google's announcement, the new service is called the Gemini Decisions API, and the model is Gemini Decision 1. It scores the proposed answers in a single pass. Claimed latency is 150–250 ms, but there is an important caveat: those measurements apply to contexts of up to 16,000 tokens. Assuming the same speed for any long document would be a mistake.

What this looks like in a product

Imagine a service that sorts customer inquiries. An email arrives with a screenshot of a charge. The developer defines three categories: billing question, technical issue and other. An additional question can ask whether the inquiry requires an urgent response. These checks concern the same material, so it makes sense to combine them in one request.

Standard product logic takes over from there. If the score for the selected category exceeds the configured threshold, the ticket automatically goes to the appropriate queue. If confidence is too low, it goes to a human agent. For example, a team might start with a 95% threshold and then adjust it based on testing. This is a hypothetical configuration example, not a value recommended by Google.

The API supports yes-or-no questions, selection from a list and scoring on a specified scale. Supported input formats include text, images, audio, video and PDF. The practical value of multimodality is clear: an email and its attached screenshot can be considered together, preserving details that matter to the decision.

Developers will still need to describe the options clearly enough. If two categories overlap, probabilities alone will not fix the classification. First, the team needs to agree on how a refund question differs from a dispute over a charge, and only then automate routing.

What to check behind the speed and pricing claims

Google lists a price of $0.10 per million input tokens, with no charge for output tokens. Shared material is billed once per request, while questions add their own instructions and criteria. The service allows up to 1000 questions for a single input; the maximum context is 256,000 tokens.

For a team, this is a reason to calculate the cost of its specific operation. How many tokens do the instructions take? How many questions are actually needed? How often will a human need to double-check the result? A cheap API call may deliver only modest savings if a large manual queue remains afterward.

The claim of a fivefold cost reduction also needs context. In the original source, Google compares the service with Gemini 3.5 Flash-Lite on a shared test suite. This is not a promise to cut every application's costs by a factor of five. Your own calculation requires your current model, real documents and identical quality requirements.

Probabilities need to be tested on your own data

The most interesting aspect of this interface is the ability to control automation through a threshold. But a 95% figure in a response still needs to be checked against the actual share of correct decisions on your inquiries. Testing rare categories and ambiguous cases separately is particularly useful: these are often where costly errors occur.

I would start with one narrow operation, such as routing tickets between departments. I would assemble a labeled sample, compare the API with the current solution and examine three metrics: routing errors, response time and the share of inquiries still handled by a human agent. Only then would I choose an operating threshold.

For a product manager, the result of that test is far more useful than a general benchmark table. It will show how many tickets can be handled automatically while keeping errors within an acceptable limit. And it will give developers a concrete basis for deciding whether to add the new service to a production system.