Machine Learning Mastery: why one vector store is not enough for AI applications

Machine Learning Mastery analyzed a common mistake in AI application architecture: treating a vector store as a complete database for the entire product. At the demo stage, this often suffices, but in production, alongside vector search, you almost always need a classical relational layer.

Where vector store excels

Vector databases have become a standard component of RAG systems because they solve a problem that ordinary SQL handles poorly: search by meaning, not by exact word matching. When a user asks a question, the system converts it into an embedding and searches for semantically closest document fragments. This allows AI to find relevant texts even when they don't contain the same phrasing as the query.

This is especially useful with unstructured data — internal documents, correspondence, PDFs, and knowledge bases. If someone asks about tenant rights when there's mold in an apartment, vector search will find sections about housing standards or landlord obligations, even if the phrase "dangerous living conditions" doesn't appear in the document. This approach better handles typos, rephrasings, and implicit context.

This is why vector store became the basic component of AI search where formulations almost never match word for word.

Where SQL is needed

The problem is that the flexibility of vector search simultaneously makes it an imprecise tool for operational tasks. It answers the question "what's semantically similar" well, but doesn't work where you need a strict answer without probabilities and tolerances. Once users, limits, payments, and object statuses appear in a product, approximate search starts to hinder rather than help. This is why in a production system, a relational database remains the place where all "hard facts" live:

• access rights and tenant boundaries, where an error becomes a data leak;
• document metadata — author, URL, upload date, file hash, publication status;
• billing, audit, logs, and any records that must be consistent;
• application state: whether a chat is archived, whether a flag is enabled, what plan a user has.

There's also another practical aspect: precise SQL filtering reduces model hallucinations. If AI must summarize only high-priority tickets closed in the last 7 days by the frontend team, you first need to strictly select exactly those records, and only then pass their text to the model. This is cheaper, faster, and safer than hoping vector search alone will randomly return the perfectly constrained dataset. Essentially, SQL here doesn't compete with the LLM, but prepares the correct working area for it.

Hybrid schema

The author proposes not choosing between the two approaches, but combining them in one data layer. A typical scenario looks like this: first, the relational database checks the user, their role, and the list of documents they have access to, and only then does the vector store search for semantic matches within this secure subset. For corporate AI assistants, this is not an optimization, but a security boundary.

Without such preliminary filtering, the system risks showing users data from another team or even another client. The reverse pattern also works. After vector search returns relevant fragments, the application can pull metadata from SQL: who authored the document, when it was updated, what trust status it has.

Then the model responds not abstractly, but with context — for example, tied to document freshness or the department that issued it. For internal knowledge bases and support agents, this noticeably increases confidence in the answer and helps users quickly verify its origin.

For teams that don't want to maintain two different databases, Machine Learning Mastery specifically highlights pgvector — a PostgreSQL extension for similarity search. In this variant, embeddings live alongside structured fields, and a single query can simultaneously check permissions, filter records by date and status, then rank them by semantic proximity. The tradeoff is simple: at moderate scale, this noticeably simplifies infrastructure, but at volumes of billions of vectors, specialized systems like Pinecone or Milvus are still faster. However, for hundreds of thousands or a few million vectors, this approach often proves to be the most pragmatic starting point.

What this means

The main conclusion is simple: vector store is an important part of the AI stack, but not a complete replacement for it. If your product works with users, access controls, payments, and system state, you can't do without a relational database. For most teams, a reasonable start is PostgreSQL with pgvector or a combination of SQL plus a separate vector database, where each technology handles only the class of tasks it solves best. The earlier this separation appears in the architecture, the less chance that the demo version will fall apart at the first real growth.