Why AI Infrastructure Has to Evolve for Agents — Akshat Bubna, Modal CTO
Modal CTO Akshat Bubna returned to the Latent Space podcast to discuss how cloud infrastructure is evolving. AI agents need a fundamentally different architecture: instead of short serverless functions, they need long-running stateful processes, memory of context and reliable scaling. Modal is building exactly that kind of platform.
AI-processed from Latent Space; edited by Hamidun News
Akshat Bubna, CTO of the Modal cloud platform, returned to the Latent Space podcast after two years to discuss why the current cloud infrastructure paradigm is ill-suited to the era of AI agents.
From stateless functions to long-running agents
For the past 15 years, cloud infrastructure has been built around short-lived functions: a request arrives, the server completes the task in milliseconds or seconds and returns a result. That model worked well for web applications, microservices and data processing.
AI agents work very differently. Instead of a one-off API request, an agent starts up and can run for hours or days, making decisions, calling tools, analyzing results and adapting to new information. It critically needs to keep state, remember context and coordinate long-term plans.
- An agent does not just call an LLM once — it iterates
- It requires a long-running process that the server does not kill
- It must remember the full context between steps
- It works with many tools, APIs and services in parallel
Why serverless architecture doesn't fit
When you run a function on AWS Lambda or Google Cloud Functions, the cloud can kill the container at any moment after execution. State is not preserved. The next function call is a blank slate.
For agents, this means every iteration loses the context of previous steps. An agent cannot work over the long term. Scaling becomes a problem: you launch many independent functions instead of managing a single coordinated, long-running process. Resources compete. Reliability suffers.
Current cloud platforms simply were not designed for this.
What agent infrastructure requires
A platform that supports agents must provide:
- Long-running containers that do not lose state
- Guaranteed memory persistence between iterations
- Simple scaling of many concurrent agents
- Built-in coordination and fault tolerance
- Easy access to GPU/CPU without having to manage infrastructure
Modal positions itself as such a platform — a cloud that first asks "what compute does the agent need?" rather than "which functions should I scale?".
What this means
Cloud infrastructure is entering the third cycle of its evolution. The first was simply about moving servers to the cloud. The second was about microservices and scaling functions. The third is the era of long-running AI agents, which require a new type of architecture. Platforms like Modal show that infrastructure must evolve alongside the types of applications developers build.
- Nov 1, 2026Google taught its StreamVibe video agent to decide when to step in on its own: +10.7%
- Nov 1, 2026OpenAI Rewrites GPT Behavior Rules: Five Values and a Chain of Command
- Nov 1, 2026Voice bots built on LLMs turn to Jev, a fast classifier, for decisions in 200–500 ms
- Nov 1, 2026Google admits AI Mode cuts search revenue, shares fall 15%
Key Questions Answered
What are the key details on From stateless functions to long-running agents?
For the past 15 years, cloud infrastructure has been built around short-lived functions: a request arrives, the server completes the task in milliseconds or seconds and returns a result. That model worked well for web applications, microservices and data processing. AI agents work very differently. Instead of a one-off API request, an agent starts up and can run for hours or days, making decisions, calling tools, analyzing results and adapting to new information. It critically needs to keep state, remember context and coordinate long-term plans. An agent does not just call
Why serverless architecture doesn't fit?
When you run a function on AWS Lambda or Google Cloud Functions, the cloud can kill the container at any moment after execution. State is not preserved. The next function call is a blank slate. For agents, this means every iteration loses the context of previous steps. An agent cannot work over the long term. Scaling becomes a problem: you launch many independent functions instead of managing a single coordinated, long-running process. Resources compete. Reliability suffers. Current cloud platforms simply were not designed for this.
What agent infrastructure requires?
A platform that supports agents must provide: Long-running containers that do not lose state Guaranteed memory persistence between iterations Simple scaling of many concurrent agents Built-in coordination and fault tolerance Easy access to GPU/CPU without having to manage infrastructure Modal positions itself as such a platform — a cloud that first asks "what compute does the agent need?" rather than "which functions should I scale?".
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.