Habr AI reveals how a single open port in an LLM platform exposed API keys and infrastructure

A story from an audit of a large RAG platform demonstrates an uncomfortable conclusion for the AI market: the most expensive incidents often begin not with a "clever" attack on the model, but with a banal infrastructure mistake. The team spent two weeks searching for complex vulnerabilities, but ultimately gained access to keys and internal services in just minutes.

How They Found Access

The auditors tested a service with tens of thousands of users and initially followed the expected path for an LLM stack: they checked SSRF in combination with LangChain, tried prompt injections, analyzed HTTP processing, and searched for deserialization vulnerabilities. The logic is understandable: these are exactly the areas usually discussed when talking about the security of RAG, agents, and LLM applications. But none of the long chains of attacks yielded results.

The critical discovery appeared not in the application, but at the level of poorly secured infrastructure.

"We made one curl to an open port — and got all the keys in 5 minutes."

This phrase captures the main contrast well. While the industry fears exotic attacks on model orchestration, real production systems often have open interfaces where you can see configs, environment variables, internal URLs, and tokens for external APIs. If such secrets sit next to working keys, compromising one service quickly turns into compromising the entire platform.

Why This Happens

LLM platforms rarely exist as a single server with one model. Typically, they consist of microservices: API gateway, chain orchestrator, vector database, file storage, queues, internal admin panels, embedding providers, logging, monitoring, and billing. Each new component adds networks, secrets, roles, and integration points. As a result, a team might have a well-configured user interface and basic authorization, but poorly isolated internal ports, debug panels, or containers with privileged access.

The problem is compounded by the fact that AI startups often think about threats at the model level. They discuss jailbreaks, system prompt leaks, hallucinations, and RAG context protection extensively, but postpone classical DevSecOps hygiene. Yet it is precisely this hygiene that determines whether an attacker can reach secrets, cloud infrastructure, or API keys worth hundreds of thousands of dollars. If an internal service listens to the network unnecessarily and secrets are available to a process "just in case," the LLM is no longer the main problem — it simply increases the cost of consequences.

Where Mistakes Occur Most Often

The material is useful because it brings the conversation about AI service security back to basic engineering discipline questions. Even if the application is built around a complex chain of retrieval, agents, and multiple models, an attacker will almost always look for the cheapest path. And this path often lies through old familiar mistakes that any team working with cloud infrastructure and containers knows about.

• Open internal ports and services without strict ACL
• Secrets in environment variables, logs, or configurations accessible to neighboring services
• Excessive permissions for containers, runners, and service accounts
• Lack of segmentation between the RAG layer, admin panel, and cloud infrastructure
• Focus on prompt injection while ignoring network hygiene

For a product, this means that security audits cannot be reduced to red teaming prompts. You need an inventory of all components around the model: which services are exposed externally, which keys are available to each container, where debug interfaces are enabled, how incoming and outgoing traffic is restricted, who has access to logs and environment snapshots. Without this, even quality protection of the application layer leaves infrastructure vulnerable, and any compromise turns from a local bug into a full-scale incident with data leaks, budget loss, and compromised access to providers.

What This Means

The main conclusion is simple: an AI product should be protected not as "special LLM magic," but as an ordinary critical platform with expensive secrets and a large blast radius. If basic infrastructure is not secured, no talk of complex model attacks will help — one open door will turn out to be more important than your entire AI architecture.