BerriAI выпустила LiteLLM Agent Platform для агентов в production

Running an AI agent in a local script is simple. Running it reliably in production, synchronizing between restarts, providing isolated environments for different contexts—that's a completely different matter.

Production Problems

When an AI agent runs only on a developer's machine, everything seems straightforward. But when moving to production, problems immediately arise: no session persistence across OS restarts, no context isolation for different users, difficult team management, no built-in monitoring. Every company that tried to run an agent in production started building its own infrastructure. BerriAI, the company behind the popular LiteLLM AI Gateway, released a solution to this problem: LiteLLM Agent Platform. It's a Kubernetes-based, self-hosted infrastructure layer that automates everything necessary.

What LiteLLM Agent Platform Includes

The platform provides key components for production-ready agents:

• Isolated sandboxes for each agent or context
• Persistent session management—state preservation across restarts
• Team management and access control
• Built-in monitoring, logging, and tracing
• Self-hosted deployment on Kubernetes
• API for managing agent lifecycle

You define the agent specification, the platform creates an environment for it and manages everything else.

Difference from LiteLLM Gateway

It's important not to confuse LiteLLM Agent Platform with LiteLLM AI Gateway (by the same author). Gateway is about managing requests to different LLM models (OpenAI, Anthropic, Cohere, local). It solves request routing and caching problems. Agent Platform is the next level up: infrastructure for the complete agent lifecycle. Gateway can be part of the stack, but Platform handles all the environment complexity.

What This Means

Practically, it eliminates an entire class of problems for teams wanting to run AI agents in production. No need to write your own session persistence, context isolation, monitoring. Being open-source code allows companies to integrate the platform into their stack without vendor lock-in. This should accelerate the transition of agents from POC to production.