Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarHabr AI→ original

Smysl.AI: An Agentic RAG Over Company Documents, Built in Kotlin Without Python

An employee asks an AI, "What is the approval period for vacation under company policy?" The answer sits in one of the company's files. Smysl.AI, built on Kotlin and LangChain4j, searches the documents in multiple rounds: if the first search misses, the agent refines its query. Only the retrieved fragments go to the language model, not the whole corpus. JVM teams get an example of how to build this kind of RAG without Python.

AI-processed from Habr AI; edited by Hamidun News
Smysl.AI: An Agentic RAG Over Company Documents, Built in Kotlin Without Python
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Two Spring Boot servicesdata-processor loads files, assistant runs the chat
Key metric #2Apache Doris handles keyword and semantic search in a single store
Key metric #3Valkey Streams passes response tokens to the server holding the SSE connection

Imagine an employee asking a corporate AI assistant: "What is the approval period for vacation under company policy?" The answer sits in one of the internal files. The system's job is to find the right fragment and answer from it, not from the "model's knowledge from the internet."

This approach is called RAG: before answering, a large language model (LLM) receives text snippets retrieved from the documents. "Agentic" RAG differs in that the model decides for itself what to search for, and can run several searches in a row if the first one fails. The author of the B2B system Smysl.

AI (smysl.ai) has published the first overview in a series of three articles. Everything is built on the JVM, meaning Kotlin, Spring Boot and the LangChain4j library, rather than the usual Python.

For Java and Kotlin teams, this is a rare breakdown of a working product rather than a tutorial.

What problem the product solves

A mid-sized or large company has plenty of internal documents, but finding the answer to a specific question takes a long time: you have to dig through folders and file storage, then ask colleagues. Many simplify this by uploading files to an external chat such as ChatGPT to "just analyze." The corpus then ends up outside the company's perimeter.

In Smysl.AI, the library is stored inside the system's own infrastructure. Only the retrieved fragments, meaning chunks or whole pages, go to the LLM, not the entire corpus. The model is connected via API. The demo uses an external provider, but the same pipeline can be pointed at a model in the company's own cloud or on-prem, provided a compatible endpoint is deployed there.

The task is deliberately narrow: a question about corporate files and an answer grounded in them, with sources cited. It is not an LMS, not a replacement for an entire knowledge base, and not an agent platform.

How it differs from ordinary RAG

Classic RAG works in a single pass: search, put all retrieved fragments into the context, answer. Here, the LLM has tools. The agent can search in multiple rounds: refine the query or pull in missing fragments, and only then answer. If the first search returns the wrong edition of a policy, this can be fixed before the answer is produced. The cost of this approach is clear: more model calls and higher latency. The author gives no measurements, so the text does not allow us to judge any accuracy gain.

Ingestion works like this: parse the file, split it into chunks, write the fragments to the index. PDF, DOCX, TXT, MD and HTML are supported; Docling handles parsing and chunking.

How the stack is built

The backend consists of two services on Kotlin and Spring Boot. The first, data-processor, handles file upload and heavy processing. The second, assistant, runs the chat, the agent and search on LangChain4j. The author explains the split by differing workloads, not by a fashion for microservices: the document pipeline and the AI layer can be developed and deployed independently.

Three decisions are worth remembering:

  • Apache Doris stores the fragments and provides full-text and semantic search in one place, without a "search engine plus vector database" combination. The author acknowledges that Qdrant also offers both modes, and that specialized databases are often stronger at pure vector search. But he values a single layer for hybrid search and headroom for load more.
  • Valkey Streams solves the streaming problem. One assistant instance generates the LLM response, while the client's SSE connection may land on another. Tokens are written to Valkey, and the instance holding the connection reads them from there.
  • GraalVM Native Image gives fast startup and lower overhead compared with a regular JVM. The price is a longer build.

The rest: PostgreSQL for metadata and chats, Keycloak for OIDC authentication, Caddy as a proxy, S3-compatible storage for files, React on the frontend, and demo hosting on VK Cloud.

Limitations and what comes next

The source code is closed and there is no public access to the demo, only screenshots and possibly a short video. The demo library is built from the documents of a fictional company. The author barely covers the backend plumbing, and describes the text itself as an account of a working product, not a turnkey implementation checklist. There are no figures yet for quality, speed or cost.

During development, the author found limitations in dependencies and filed issues in Docling and Docling Java, as well as a PR in LangChain4j.

The next two articles cover the document-processing pipeline from Docling to the Doris index, and the agent: tools, corpus search, answers with sources, and token streaming. For JVM teams, this is a chance to see concrete solutions before they decide to rewrite everything in Python.

Story

Key Questions Answered

What problem the product solves?

A mid-sized or large company has plenty of internal documents, but finding the answer to a specific question takes a long time: you have to dig through folders and file storage, then ask colleagues. Many simplify this by uploading files to an external chat such as ChatGPT to "just analyze." The corpus then ends up outside the company's perimeter. In Smysl.AI, the library is stored inside the system's own infrastructure.

How it differs from ordinary RAG?

Classic RAG works in a single pass: search, put all retrieved fragments into the context, answer. Here, the LLM has tools. The agent can search in multiple rounds: refine the query or pull in missing fragments, and only then answer. If the first search returns the wrong edition of a policy, this can be fixed before the answer is produced. The cost of this approach is clear: more model calls and higher latency. The author gives no measurements, so the text does not allow us to judge any accuracy gain.

How the stack is built?

The backend consists of two services on Kotlin and Spring Boot. The first, data-processor, handles file upload and heavy processing. The second, assistant, runs the chat, the agent and search on LangChain4j. The author explains the split by differing workloads, not by a fashion for microservices: the document pipeline and the AI layer can be developed and deployed independently.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…