Runpod shows how to spin up your own API on Serverless in four steps

Runpod has published a step-by-step guide to building your own API on its Serverless platform. Going from an empty folder to a working HTTP endpoint takes four steps: write a worker, test it locally, build a Docker image, and create an endpoint in the console. The platform adds five service routes, a request queue and autoscaling on its own, and you pay only for the seconds a worker spends processing a job.

The material is introductory and describes no new features. But for teams that want to move model inference into a separate service without standing up a cluster, it is a convenient entry point.

How a worker is built

The core of the whole setup is the handler function. It receives a dictionary describing the job, and the input data sits under the input key. Inside, you can do whatever you need: load a model into GPU memory when the container starts, generate text, process an image, or simply add two numbers as a sanity check. The function returns its result as an ordinary Python object, and the platform serializes it to JSON.

To turn the function into a worker, you pass it to runpod.serverless.start({"handler": handler}). This is the only point of integration with the SDK. Heavy objects, such as model weights, are best initialized outside the handler, at module level. That way they load once when the worker starts, not on every request.

You can test the code before deploying. If you put a test_input.json file next to the script and run it, the SDK runs the handler on that test input and prints the result to the console. There is also a local server mode with the same routes as in the cloud, which you can call with curl or Postman.

Endpoints and invocation modes

Once the endpoint is created, Runpod issues an ID and a public address in the form api.runpod.ai/v2/{id}. Requests are authorized with an API key in a Bearer header. The set of routes is fixed:

• /run: asynchronous launch; the response returns a job id;
• /runsync: synchronous call; the connection stays open until the result is ready;
• /status/{job_id}: status and result of an asynchronous job;
• /cancel/{job_id}: cancels a job that is queued or running;
• /health: state of the workers and the queue.

Job duration determines the choice between /run and /runsync. Short operations are easier to call synchronously. Image generation, transcription of long audio, and anything else that can take minutes is better sent asynchronously, with status polling. For streaming responses, such as token-by-token output from a language model, you can make the handler a generator.

Deployment and cost

The code and dependencies are packaged into a Docker image and published to a registry. When creating the endpoint in the console, you specify the image, the GPU or CPU type, the worker limit and the timeouts. From there, the platform starts and stops containers on its own based on queue length.

There are two modes. Flex workers start on demand and shut down when idle, so you do not pay for idle time, but the first request after a pause waits for a cold start. For models with large weights, this can be noticeable. Active workers run continuously: latency is lower and the per-second price is lower too, but they are billed as busy even with no traffic. Billing is per second.

Where to start

A minimal set of actions is enough for a first experiment:

1. Install the runpod package and write a handler that returns a simple result. 2. Run it locally through test_input.json and make sure the input and output format works for the client. 3. Build the image, push it to a registry, and create an endpoint with one worker and a short timeout. 4. Call /runsync with curl, then move on to /run and /status. 5. Only then plug in a real model and measure the cold start.

It is best to measure the time to first response after idle right away and compare it against the product's requirements. That number determines the choice between flex and active workers. Image size also affects cold start, so large model weights are better kept in network storage or cached rather than baked into the image without need.