Habr AI breaks down how one open port in an LLM platform led to keys and infrastructure

A story from an audit of a large RAG platform points to an uncomfortable conclusion for the AI market: the costliest incidents often begin not with a clever attack on the model, but with a mundane infrastructure mistake. The team spent two weeks hunting for sophisticated vulnerabilities, but in the end got access to keys and internal services in literally a few minutes.

How they found access

The auditors tested a service with tens of thousands of users. They first followed the route you would expect for an LLM stack: they checked for SSRF in combination with LangChain, tried prompt injections, analyzed HTTP handling and looked for deserialization vulnerabilities. The logic is clear: these are the areas usually discussed when the topic is the security of RAG, agents and LLM applications. But none of the long chains produced a working result. The critical finding came not in the application, but at the level of poorly secured infrastructure.

"We made one curl request to an open port and got all the keys in 5 minutes."

That line captures the main contrast of the piece. While the industry fears exotic attacks on model orchestration, real production environments often still have open interfaces that expose configs, service environment variables, internal URLs and tokens for external APIs. If such secrets sit next to production keys, compromising one service quickly turns into compromising the entire platform.

Why this happens

LLM platforms rarely run as a single server with a single model. Usually they are a set of microservices: an API gateway, a chain orchestrator, a vector database, file storage, queues, internal admin panels, embeddings providers, logging, monitoring and billing. Every new component adds network surface, secrets, roles and integration points. As a result, a team may have a well-configured user interface and basic authorization, while internal ports, debug panels or containers with privileged access are poorly isolated.

The problem is compounded by the fact that AI startups often think about threats at the model level. They talk a lot about jailbreaks, system prompt leaks, hallucinations and protecting RAG context, but put off classic DevSecOps hygiene. Yet that hygiene determines whether an attacker can reach secrets, cloud infrastructure or paid keys worth hundreds of thousands of dollars. If an internal service listens on the network without need, and secrets are available to a process "just in case", the LLM is no longer the main problem here; it merely raises the cost of the consequences.

Where mistakes are most common

The material is useful because it brings the conversation about AI service security back to the basics of engineering discipline. Even if an application is built around a complex retrieval chain, agents and several models, an attacker almost always looks for the cheapest path. And that path often runs through old, familiar mistakes known to any team that has worked with cloud infrastructure and containers.

• Open internal ports and services without strict ACLs
• Secrets in env, logs or configuration accessible to neighboring services
• Excessive privileges for containers, runners and service accounts
• No segmentation between the RAG layer, the admin panel and cloud infrastructure
• Focus on prompt injection while ignoring network hygiene

For a product, this means a security audit cannot be reduced to red teaming prompts. You need an inventory of all components around the model: which services are exposed externally, which keys each container can access, where debug interfaces are enabled, how outbound and inbound traffic is restricted, and who has access to logs and environment snapshots. Without this, even strong protection of the application layer leaves the infrastructure vulnerable, and any compromise turns from a local bug into a full-scale incident with leaked data, budget and access to providers.

What this means

The main takeaway is simple: an AI product should be protected not as "special LLM magic" but as an ordinary critical platform with expensive secrets and a large blast radius. If the basic infrastructure is not locked down, no talk of sophisticated attacks on the model will help: one open door turns out to matter more than your entire AI architecture.