Charoite Developer Explains Where an NLI Model Works in a Zoom Assistant and Where It Doesn't

In part ten of his series on a local assistant for Zoom calls, the author of the Charoite ("Чароит") project breaks down the NLI model that has been running in his system for three months. The announcement on Habr opens with an example that shows why it is needed: "the budget for the quarter is approved" and "the budget for the quarter is not approved" are almost the same phrase to embeddings. The words match, and the particle "not" carries little weight in the vector.

For a meeting, it means the opposite was decided. The announcement does not include the full text, so there are no figures or benchmarks below: only what the author stated directly, and the context in which to read it.

Why embeddings confuse opposites

An embedding model compresses a sentence into a vector and measures closeness by topic and vocabulary. That works well for searching a transcript: the query "what was decided about the budget" will find the right passage. But the same mechanism reacts weakly to negation, conditions, and shifts in modality. "Approved," "not approved," and "we'll approve it if we find the money" end up as neighbors in vector space, even though for the minutes they are three different facts.

NLI (natural language inference) solves a different problem. The model takes a pair of statements and determines whether the second follows from the first, contradicts it, or is unrelated to it. This is the classic three-class setup: entailment, contradiction, neutral. In this setup, the pair about an approved and a not-approved budget yields contradiction, not "very similar texts."

What we know about Charoite and NLI

Charoite is a local assistant for calls. The author is writing a series of articles about it, and part ten is devoted entirely to a single component. The title and introduction reveal three things.

• The NLI model has been running in Charoite for three months, so it is not a weekend experiment but a component with an operating history.
• The author tried several places to apply it: the title mentions three where the model was "tried on" and two where it was dropped.
• The article promises a separate look at the price of the decision: the author writes that removing the model "cost something."

The announcement does not name the model, its size, latency, or accuracy. For a local assistant, these are key parameters: an NLI classifier is called on pairs of statements, and the number of pairs grows quickly in a long meeting. If you compare every new claim with all the previous ones, the load becomes quadratic, which is noticeable on home hardware.

Where NLI usually fits and where it doesn't

Judging by the title, the author shows both sides in the full text. From general practice we can guess which uses fall into each group, but that is a guess, not a summary of the article. NLI suits checks on pairs where an error is expensive: detecting contradictions between decisions made at different meetings, checking the final minutes against the transcript, and verifying that a point in the summary was actually said. It works worse where fine-grained interpretation of a long context is needed. Spoken language with hedges, unfinished phrases, and references like "as we decided yesterday" does not map well onto pairs of short statements. The classifier is trained on clean sentences, not on live transcripts.

That points to a typical reason for pulling NLI from a task: false positives on noisy transcription cost more than missed contradictions. Whether the author confirms this particular reason will become clear from his article.

How to use this breakdown

If you are building your own meeting assistant or transcript search, three things are worth doing.

• Assemble a small set of pairs with negations and conditions ("approved" versus "not approved," "if" versus "when") and run them through your own embedding search. You will see where the vector gets it wrong.
• Put NLI after candidate selection, not in place of it: embeddings quickly find similar statements, and NLI checks only those pairs.
• Measure quality on real transcripts, not on clean test sentences. That is where the model most often ends up being removed.

Part ten of the series is interesting as a rare report on three months of operation rather than on a launch. Specific conclusions, figures, and the cost of dropping the model should be taken from the original article on Habr.