JEV replaces multistep generation with a single typed decision

A large language model inside an AI agent often works like a professor asked to sort inquiries into three folders. It receives the task history, rereads the context, and generates an answer sequentially—even though the system only needs one choice: run a search, check an API, or hand the request to a human.

JEV from TypeSafe AI proposes separating these short decisions from text generation. The model accepts the system state and a set of predefined questions, then returns typed answers: a selected option, a score on a scale, or the probability of a “yes.” For developers, the important point is not new magic but an architectural pattern: the expensive LLM handles analysis and text, while a compact component handles routing.

How it works in an agent loop

Consider support for an online store. A customer writes: “Stripe has been unable to connect for three days, and I am losing sales.” The agent needs to determine the urgency and the next step.

A conventional LLM might receive a system prompt, tool descriptions, and a JSON schema, then generate an object such as urgent along with an explanation. JEV is structured differently: the developer supplies context and questions with expected types. The Choice primitive selects one option from a list, Score assigns a rating, and Noul returns the probability that a statement is true. Several independent questions about the same context can be processed in a single request.

The resulting output immediately feeds into ordinary program logic. If the probability of a critical incident exceeds a set threshold, the ticket goes to an operator. If it is lower, the agent searches the knowledge base for an answer. In the borderline range, the system can call a large LLM or request more information.

This component does not replace the entire agent. It handles a narrow fork within the “observation—decision—action” loop, after which the API, RAG, tools, or generative model continue the work.

Where the practical benefit appears

The main candidates are operations with a limited set of valid outcomes: ticket classification, tool selection, urgency assessment, and determining whether escalation is necessary. Free-form generation is genuinely redundant here.

The benefit has three parts. First, the response has a predefined type, so there is no need to extract JSON from text. Second, several assessments can be requested together. Third, the large model is called only where reasoning, synthesis, or polished writing is required.

JEV should not be confused with RAG or a reranker. RAG is a pattern for obtaining external context. A reranker sorts retrieved documents by relevance to the query. JEV selects an action or evaluates a state. All three components can operate in one system: JEV decides to consult the knowledge base, RAG finds passages, and the reranker selects the best ones.

What to verify before deployment

A typed response is not necessarily a correct response. A figure of 92% looks convincing, but without calibration on your data, it may poorly reflect the actual error rate. It is especially dangerous to connect such a score automatically to payments, account suspensions, or other irreversible actions.

Start in shadow mode: record JEV’s decisions, but do not execute them automatically. Compare the model with simple rules, a compact LLM, and the current router on accuracy, latency, cost, and escalation rate. Measure errors in rare and costly scenarios separately.

If an if/else statement solves the task reliably, a neural network is unnecessary. If there are few options but the rules keep expanding and depend on the meaning of the text, a specialized model becomes interesting. The practical conclusion is simple: JEV should be viewed not as an “LLM killer,” but as a narrow router that frees the large model from routine decisions.