AI Agents in Production: An Enterprise AI Harness Architecture on Kubernetes
An engineer has published an architecture on Habr for running AI agents in production. Instead of writing a custom runtime, as Anthropic did, the author assembled the infrastructure from open components so that ordinary platform teams can deploy agents to Kubernetes. The reference architecture has four functional layers, with clearly defined integration points between them.
AI-processed from Habr AI; edited by Hamidun News
An engineer has published a reference architecture on Habr for an Enterprise AI Harness on Kubernetes. Rather than write a custom runtime, the author built the infrastructure from open components, so that any platform team can run production agents, not just Anthropic.
Why are production agents a blind spot?
The community offers dozens of articles and courses on how to design, train or configure an AI agent. Far less is known about how to run one in production safely, reliably and predictably. That knowledge usually sits behind the closed doors of companies such as Anthropic, OpenAI or Google, which have the resources and expertise to write their own runtime from scratch. The work involves hard engineering problems: state management, fault tolerance, context isolation, monitoring and security.
Which layers make up the architecture?
The author proposes a reference architecture split into four functional layers:
- Orchestration layer: agent lifecycle management, task scheduling, error handling and retry policies, request queue management
- Integration layer: connections to external services, REST APIs, tools and data stores
- Execution layer: the actual running of the agent, context and memory management, calls to LLM APIs
- Observability layer: logging of all operations, performance metrics, distributed tracing, alert configuration
Clear contractual integration points are defined between the layers, so individual components can be replaced without rewriting the whole system. This matters in production: you cannot bet the entire stack on a single implementation. For example, if an orchestration tool is unsuitable for some reason, or you need to integrate with an existing system, you can pick an alternative without rewriting the integration or execution layer.
Who is this architecture for?
The solution targets platform teams at large companies or ambitious startups that want to:
- Let developers deploy AI agents on their own, without having to build infrastructure from scratch
- Avoid lock-in to cloud services such as Azure AI Services or AWS Bedrock
- Guarantee reliability, security and predictability in production
The solution runs on Kubernetes, the industry standard for containerization in the enterprise. This means most existing DevOps tooling (CI/CD pipelines, network policies, backup strategies) will work out of the box without rework.
What this means
Production deployment of AI agents is moving from experimentation to standard engineering practice. Open reference architectures and best practices lower the barrier to entry and speed up adoption for teams that are unable or unwilling to write their own runtime from scratch. This matters most in the enterprise, where reliability and security compliance rank above prototyping speed.
- Nov 1, 2026OpenAI Rewrites GPT Behavior Rules: Five Values and a Chain of Command
- Nov 1, 2026Google admits AI Mode cuts search revenue, shares fall 15%
- Oct 31, 2026Author puts his script before an AI jury: the verdict cost 850 thousand tokens
- Oct 31, 2026Anthropic: Fold Subagents Back Into a Single Loop — One Customer's Tasks Ran Twice as Fast
Key Questions Answered
Why are production agents a blind spot?
The community offers dozens of articles and courses on how to design, train or configure an AI agent. Far less is known about how to run one in production safely, reliably and predictably. That knowledge usually sits behind the closed doors of companies such as Anthropic, OpenAI or Google, which have the resources and expertise to write their own runtime from scratch. The work involves hard engineering problems: state management, fault tolerance, context isolation, monitoring and security.
Which layers make up the architecture?
The author proposes a reference architecture split into four functional layers: Orchestration layer: agent lifecycle management, task scheduling, error handling and retry policies, request queue management Integration layer: connections to external services, REST APIs, tools and data stores Execution layer: the actual running of the agent, context and memory management, calls to LLM APIs Observability layer: logging of all operations, performance metrics, distributed tracing, alert configuration Clear contractual integration points are defined between the layers, so individ
Who is this architecture for?
The solution targets platform teams at large companies or ambitious startups that want to: Let developers deploy AI agents on their own, without having to build infrastructure from scratch Avoid lock-in to cloud services such as Azure AI Services or AWS Bedrock Guarantee reliability, security and predictability in production The solution runs on Kubernetes, the industry standard for containerization in the enterprise. This means most existing DevOps tooling (CI/CD pipelines, network policies, backup strategies) will work out of the box without rework.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.