Smysl.AI: An Agentic RAG Over Company Documents, Built in Kotlin Without Python
An employee asks an AI, "What is the approval period for vacation under company policy?" The answer sits in one of the company's files. Smysl.AI, built on Kotlin and LangChain4j, searches the documents in multiple rounds: if the first search misses, the agent refines its query. Only the retrieved fragments go to the language model, not the whole corpus. JVM teams get an example of how to build this kind of RAG without Python.
AI-processed from Habr AI; edited by Hamidun News
Imagine an employee asking a corporate AI assistant: "What is the approval period for vacation under company policy?" The answer sits in one of the internal files. The system's job is to find the right fragment and answer from it, not from the "model's knowledge from the internet."
This approach is called RAG: before answering, a large language model (LLM) receives text snippets retrieved from the documents. "Agentic" RAG differs in that the model decides for itself what to search for, and can run several searches in a row if the first one fails. The author of the B2B system Smysl.
AI (smysl.ai) has published the first overview in a series of three articles. Everything is built on the JVM, meaning Kotlin, Spring Boot and the LangChain4j library, rather than the usual Python.
For Java and Kotlin teams, this is a rare breakdown of a working product rather than a tutorial.
What problem the product solves
A mid-sized or large company has plenty of internal documents, but finding the answer to a specific question takes a long time: you have to dig through folders and file storage, then ask colleagues. Many simplify this by uploading files to an external chat such as ChatGPT to "just analyze." The corpus then ends up outside the company's perimeter.
In Smysl.AI, the library is stored inside the system's own infrastructure. Only the retrieved fragments, meaning chunks or whole pages, go to the LLM, not the entire corpus. The model is connected via API. The demo uses an external provider, but the same pipeline can be pointed at a model in the company's own cloud or on-prem, provided a compatible endpoint is deployed there.
The task is deliberately narrow: a question about corporate files and an answer grounded in them, with sources cited. It is not an LMS, not a replacement for an entire knowledge base, and not an agent platform.
How it differs from ordinary RAG
Classic RAG works in a single pass: search, put all retrieved fragments into the context, answer. Here, the LLM has tools. The agent can search in multiple rounds: refine the query or pull in missing fragments, and only then answer. If the first search returns the wrong edition of a policy, this can be fixed before the answer is produced. The cost of this approach is clear: more model calls and higher latency. The author gives no measurements, so the text does not allow us to judge any accuracy gain.
Ingestion works like this: parse the file, split it into chunks, write the fragments to the index. PDF, DOCX, TXT, MD and HTML are supported; Docling handles parsing and chunking.
How the stack is built
The backend consists of two services on Kotlin and Spring Boot. The first, data-processor, handles file upload and heavy processing. The second, assistant, runs the chat, the agent and search on LangChain4j. The author explains the split by differing workloads, not by a fashion for microservices: the document pipeline and the AI layer can be developed and deployed independently.
Three decisions are worth remembering:
- Apache Doris stores the fragments and provides full-text and semantic search in one place, without a "search engine plus vector database" combination. The author acknowledges that Qdrant also offers both modes, and that specialized databases are often stronger at pure vector search. But he values a single layer for hybrid search and headroom for load more.
- Valkey Streams solves the streaming problem. One assistant instance generates the LLM response, while the client's SSE connection may land on another. Tokens are written to Valkey, and the instance holding the connection reads them from there.
- GraalVM Native Image gives fast startup and lower overhead compared with a regular JVM. The price is a longer build.
The rest: PostgreSQL for metadata and chats, Keycloak for OIDC authentication, Caddy as a proxy, S3-compatible storage for files, React on the frontend, and demo hosting on VK Cloud.
Limitations and what comes next
The source code is closed and there is no public access to the demo, only screenshots and possibly a short video. The demo library is built from the documents of a fictional company. The author barely covers the backend plumbing, and describes the text itself as an account of a working product, not a turnkey implementation checklist. There are no figures yet for quality, speed or cost.
During development, the author found limitations in dependencies and filed issues in Docling and Docling Java, as well as a PR in LangChain4j.
The next two articles cover the document-processing pipeline from Docling to the Doris index, and the agent: tools, corpus search, answers with sources, and token streaming. For JVM teams, this is a chance to see concrete solutions before they decide to rewrite everything in Python.
- Nov 19, 2026Anthropic goes public and raises $81.3 billion, beating SpaceX's $75 billion record
- Nov 19, 2026Infostart connects Direct, Metrica and Wordstat to ChatGPT: 6 commands, shared access
- Nov 19, 2026Amazon and Google's Anthropic Stakes Valued at $308 Billion in IPO at $83 a Share
- Nov 19, 2026Google's Gemini 4 Neon runs twice as fast as Carbon and costs 17% less per task
Key Questions Answered
What problem the product solves?
A mid-sized or large company has plenty of internal documents, but finding the answer to a specific question takes a long time: you have to dig through folders and file storage, then ask colleagues. Many simplify this by uploading files to an external chat such as ChatGPT to "just analyze." The corpus then ends up outside the company's perimeter. In Smysl.AI, the library is stored inside the system's own infrastructure.
How it differs from ordinary RAG?
Classic RAG works in a single pass: search, put all retrieved fragments into the context, answer. Here, the LLM has tools. The agent can search in multiple rounds: refine the query or pull in missing fragments, and only then answer. If the first search returns the wrong edition of a policy, this can be fixed before the answer is produced. The cost of this approach is clear: more model calls and higher latency. The author gives no measurements, so the text does not allow us to judge any accuracy gain.
How the stack is built?
The backend consists of two services on Kotlin and Spring Boot. The first, data-processor, handles file upload and heavy processing. The second, assistant, runs the chat, the agent and search on LangChain4j. The author explains the split by differing workloads, not by a fashion for microservices: the document pipeline and the AI layer can be developed and deployed independently.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.