AI for Business: MTS Open-Sources AVI, a Filter to Protect LLMs

MTS has released AVI, an open-source service for deploying AI safely in companies. It is a filter for large language models (LLMs) that controls requests and responses without changing the model's weights.

What AVI is and how it works

AVI is a two-level "smart firewall" between the user and the language model. The input filter intercepts prompt attacks and unwanted requests before they reach the model. The output filter checks the response after it has been generated, assessing toxicity and compliance with legislation and ethical norms.

Key components of the published solution:

• A service built on Python and FastAPI, the backbone of the architecture
• A RAG module for context-aware checking of filtering rules
• A Docker build with a full monitoring stack: Prometheus, Grafana, Jaeger
• Tooling for reproducing experiments on the FinanceBench benchmark
• A scientific description published in the peer-reviewed journal MDPI.Electronics

What sets the system apart is how rules are added. Instead of editing configs or retraining the model, you define a new filter with a single natural-language sentence. The RAG module picks it up and applies it on the next check.

Why filtering from outside beats retraining

The standard way to make a language model "law-abiding" is to further train it with RLHF or fine-tuning. That is expensive, takes weeks, and requires a full new cycle whenever requirements change: a new regulation comes out or company policy shifts, and you retrain.

AVI takes a fundamentally different approach: the filter lives outside the model and handles traffic as a proxy layer. Need to ban discussion of competitors? Just add a rule to the config. Need to account for 152-FZ or GDPR requirements? Same thing. The model itself stays untouched.

"AVI works as a smart firewall for LLMs: it repels prompt attacks at

the input and checks responses for toxicity, ethics and violations of the law at the output," the authors say of the concept.

The authors claim this approach delivers noticeable savings. For companies in regulated industries such as banking, telecom and healthcare, that could mean months of ML team work saved: instead of retraining, it is enough to update the filter's rule set.

What the FinanceBench tests showed

The team tested the system on FinanceBench, a standard set of financial questions that require accurate and legally correct answers. The results were published in the peer-reviewed journal MDPI.Electronics.

Detailed accuracy metrics are available in the full text of the scientific paper in English. Publication in a peer-reviewed journal and the release of the code mean the system has undergone independent review and did not remain an internal company tool.

What this means

MTS is releasing a practical tool for anyone deploying LLMs in production environments with strict content requirements. The "firewall instead of fine-tuning" approach may become a standard for corporate AI deployments, especially where regulatory requirements change faster than a model can be retrained.

Frequently asked questions

Does AVI work with any language model?

Yes. The filter does not depend on the architecture or weights of a particular model. AVI connects as a proxy layer and can work with any LLM service.

What does the published repository include?

The repository includes a Python/FastAPI backend, a RAG module for handling natural-language rules, a Docker build with monitoring via Prometheus, Grafana and Jaeger, and tools for reproducing experiments on FinanceBench.

What is AVI and how does it protect AI?

AVI is a two-level filter for language models. The input filter intercepts dangerous requests before they reach the model. The output filter checks the generated response for compliance with legislation and ethical norms.