Amazon SageMaker AI removed mandatory S3 upload for asynchronous inference

Amazon updated SageMaker AI: now asynchronous inference accepts input data directly in the API request body — without mandatory pre-upload to Amazon S3.

How Asynchronous Inference in SageMaker Works

Async Inference is a mode for ML tasks that cannot fit within a few seconds of wait time. Audio transcription, PDF document processing, long text generation, batch image classification — all these are scenarios where you have to wait from tens of seconds to several minutes for a model response. Synchronous inference does not work in such cases: the client cannot keep an HTTP connection open for that long.

Asynchronous mode solves this through a queue: the request is queued for processing, the model works in the background, the result is saved to an S3 bucket, and the client receives a notification or periodically checks for readiness. This is the standard pattern for production ML systems with variable response times.

Until today's update, the standard workflow required two steps: first upload input data to S3, then pass a link to that object to `InvokeEndpointAsync`. Only then would SageMaker begin processing. Each call meant an extra write operation to storage and an additional network request.

What Changed Now

Amazon added support for inline payloads: data can now be passed directly in the body of the `InvokeEndpointAsync` request — like in a regular REST API. Pre-uploading input data to S3 is no longer required.

Practical changes:

• Less code: no need to write logic for creating S3 objects, passing keys, and cleaning up temporary files
• Lower operational costs: each PUT operation in S3 costs money — with thousands of small calls per day this adds up to a significant sum
• Simpler debugging: input data is visible directly in the request, no need to go to S3 Console to understand what exactly was passed to the input
• Fewer points of failure: one network call instead of two reduces the likelihood of failure during the data upload phase
• Faster processing startup: SageMaker begins inference immediately, without waiting to fetch data from storage

Scenarios Where This Update is Relevant

The update is especially useful in scenarios where input data is small to medium in size and is generated dynamically.

Embedded ML requests from applications: when a user sends text for analysis or an image for classification, storing an intermediate object in S3 just to pass it to the model is extra work. Now data goes directly.

Testing after deployment: in CI/CD pipelines, model verification after release may include dozens of test requests. Managing temporary S3 objects for each of them becomes unnecessary.

For heavy data — long video files, large audio recordings, document corpora — the path through S3 remains optimal and available. Both approaches work in parallel, and the developer chooses the one suitable for the specific task.

What This Means

Amazon is consistently removing infrastructure barriers in SageMaker. Support for inline payloads is a targeted but concrete improvement: fewer steps, fewer dependencies, cleaner pipeline architecture. For teams with high-frequency asynchronous inference, this is a real savings in development time and operational costs.