Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarAWS Machine Learning Blog→ original

AgentCore V2 from AWS launches agents from memory snapshots, but for chat and voice even 2 seconds is too long

Previously, every new AI agent session in Amazon Bedrock AgentCore started its container from scratch, and startup time grew with the size of the image. In V2, the environment is prepared once, frozen as a snapshot of memory and disk, and restored for each new session, much like loading a saved game: imports and clients are already in place. Most agents are fine after switching to V2, but for chat and voice even a 2-second startup is too much. That is why you should move all initialization ahead of app.run().

AI-processed from AWS Machine Learning Blog; edited by Hamidun News
AgentCore V2 from AWS launches agents from memory snapshots, but for chat and voice even 2 seconds is too long
Source: AWS Machine Learning Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceAWS Machine Learning Blog ↗
Editor-in-Chief
Key metric #1Building the snapshot takes several minutes on every create or update
Key metric #2Files from the image are read lazily, and the file cache is cleared before freezing
Key metric #3Open network connections go stale after restore, so close them in advance

Picture an agent that processes insurance claims or reviews code. If it runs in the background, nobody notices how many seconds it takes to warm up. But put it behind a chat window or a voice assistant, and the pause before the first response becomes part of the product.

AWS has described how fast session startup works in Amazon Bedrock AgentCore runtime in platform version V2 (the platformVersion parameter set to "V2") and how to restructure an agent to get the most out of it. The idea: instead of starting a container from scratch for every user, the platform prepares the environment once, saves a snapshot of it, and unpacks that snapshot into each new session. According to AWS's estimate, a new session takes roughly 2 seconds for many scenarios, but for chat and voice even that is too slow.

How V2 starts a session

Each session in AgentCore runs in its own isolated microVM. In V1, on every new launch the platform pulled the container image from Amazon ECR, started the container, and ran your initialization code. Most of the time went to downloading the image, so cold start grew with its size.

In V2, this work is done once, when the runtime is created or updated. AgentCore starts the container in a clean microVM and waits for the first successful response to GET /ping, for no more than 120 seconds. It then clears the operating system's file cache to keep the snapshot small, freezes the microVM, and saves its memory and disk. When a new session arrives, the snapshot is restored into a new microVM, and the process continues from the point where it was frozen, with imports, objects, and clients already ready. The trade-off: every create or update now takes several minutes while the snapshot is built.

Three rules follow from this. Memory comes back in full, including files on RAM disks such as /dev/shm. Reading a file during initialization gains nothing, because the cache is cleared before freezing. And files from the image are loaded lazily: each block is fetched from snapshot storage on first access.

What to move into initialization

The classic cold-start advice is to import lazily and keep initialization short. In V2 it is the opposite. Initialization runs once, when the snapshot is built, while work deferred to the request handler is paid for by every new session. By "initialization," AWS means everything the agent does before /ping returns healthy; in the Python SDK, that is everything before app.run().

AWS's recommendations:

  • create objects, not just import modules, including local tools that depend on external libraries;
  • create clients, for example boto3: endpoint and credential resolution comes back with the snapshot, and sockets reconnect on the first call. Some other network clients try a stale socket first, which adds hundreds of milliseconds, so close their connections before the snapshot;
  • keep static data in memory: prompts, configuration, small indexes. Reading a file is not enough; store the result in a variable;
  • run long-lived helpers, such as a language server, and communicate with them over a pipe or socket, so a subprocess does not read binaries from the image in every session;
  • copy files that cannot stay in memory, such as the tsc compiler, into /dev/shm;
  • ship precompiled bytecode: set UV_COMPILE_BYTECODE=1 or use pip install --compile.

If an agent needs large files, a CLI, or a browser, and cannot keep them in its own process, AgentCore now supports build hooks: they download and "warm up" these resources while the snapshot is being built. For large reference data that is read selectively, AWS recommends mounting Amazon S3 Files or Amazon EFS through an access point (VPC mode required).

How to measure it and what it costs

Latency in an agent application consists of platform time (session preparation) and your code's time (model calls, tools, subagents, streaming the response). They are fixed differently, so first you need to understand what dominates. In CloudWatch (the AWS/Bedrock-AgentCore namespace), Latency, meaning time to first byte, and OverheadLatency, meaning the platform's share, are available. Your code's time is the difference between them. Initialization time on V2 is not visible in requests: you need to log it yourself once, before app.run().

Look at the P90 and P99 percentiles, or the trimmed mean TM90, which discards the slowest 10% of requests: a simple average hides slow cases. Test at a realistic share of new sessions, not with a few runs right after deployment.

There is also an honest trade-off: everything you keep in memory inflates the snapshot, and that memory is paid for in every session. So measure total session startup before and after changes, to make sure the latency has not simply moved from the handler into restore. If the agent mostly calls a cloud model, V2 and careful initialization are usually enough; for interactive chat and voice, AWS additionally suggests starting the session in the background as soon as the user opens the app or chat window. The source material describes this technique only briefly, so no further details are given here.

Story

Key Questions Answered

How V2 starts a session?

Each session in AgentCore runs in its own isolated microVM. In V1, on every new launch the platform pulled the container image from Amazon ECR, started the container, and ran your initialization code. Most of the time went to downloading the image, so cold start grew with its size. In V2, this work is done once, when the runtime is created or updated. AgentCore starts the container in a clean microVM and waits for the first successful response to GET /ping, for no more than 120 seconds.

What to move into initialization?

The classic cold-start advice is to import lazily and keep initialization short. In V2 it is the opposite. Initialization runs once, when the snapshot is built, while work deferred to the request handler is paid for by every new session. By "initialization," AWS means everything the agent does before /ping returns healthy; in the Python SDK, that is everything before app.run().

How to measure it and what it costs?

Latency in an agent application consists of platform time (session preparation) and your code's time (model calls, tools, subagents, streaming the response). They are fixed differently, so first you need to understand what dominates. In CloudWatch (the AWS/Bedrock-AgentCore namespace), Latency, meaning time to first byte, and OverheadLatency, meaning the platform's share, are available. Your code's time is the difference between them.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Private AI Builders Community · Vibe Club

Ship AI Faster with Founders, Engineers & Weekly Architecture Reviews

Inside Vibe Club (hamidun.com/club) — weekly live masterminds with Zhemal Khamidun, Tech Radar, full LMS access, and 370+ production Claude Code skills.

Join Vibe Club →Explore Hamidun Academy
What do you think?
Loading comments…