OpenAI built Pando, a Git backend that made pushes 5x faster under Codex load

Imagine that every engineer at a company suddenly gained a dozen assistants, each pushing code, opening branches, and running CI. That is exactly what happened at OpenAI when Codex began writing code at scale. The monorepository's bottleneck was no longer people but the Git server itself.

What broke

Since late last year, the systems around the monorepo had stopped keeping up: slow fetches and pushes, failed merges, and reliability sometimes below 90%. Load grows multiplicatively: one engineer keeps several changes in CI, while every push launches multiple jobs that fetch the same commit.

The engineers did not abandon Git. They decided to retain the protocol and client so that people and agents would not have to relearn anything and improvements could be contributed upstream. They replaced only the backend and did so without downtime. That is how Pando emerged.

Why not 3PC or WAL

Traditional approaches to scaling Git run into a single coordination point.

The first option is full replicas and three-phase commit. Every push requires several rounds of communication between the coordinator and replicas, while replica disks and networks enter the critical write path. Refs are updated one at a time.

The second option is a shared manifest in blob storage with a write-ahead log. Packs are uploaded in parallel, but every update still modifies one manifest through compare-and-swap. Pushes to unrelated branches contend with one another. Batching helps, but adds latency.

How Pando works

Its authors discarded two conventional assumptions.

First: repository-wide consistency. Pando stores an incoming packfile as an immutable blob and places its metadata—the new branch tip, pack, and dependencies on earlier ref versions—in horizontally scalable storage. Each branch is updated independently, with serializability within a single ref. This provides causal consistency: a reader sees the history of whoever wrote the ref, but not a single snapshot of the entire repository.

In practice, this means that when branches foo and bar are being updated during a clone, one client may see foo version 1 and bar version 2, while another sees the reverse. The authors argue that people and agents care about the update order within one branch, not a global order.

Second: a unique pack for every client. A conventional server builds a custom package for each client to avoid sending unnecessary data, consuming CPU and disk resources. Pando reuses packfiles: a client may download some objects it already has, but the server can serve the same bytes to many clients with almost no computation. The read-side goal is to saturate all network bandwidth available to the client.

Results and limitations

In production, median push latency fell by nearly 5x, and Pando serves almost a petabyte of data per day. The trade-off is clear: there is no global snapshot of all refs, and clients that already have some objects receive extra traffic.

The article also includes a 1,000-client simulation comparing three models, but the authors base implementation comparisons on models rather than independent benchmarks. Pando's code has not been published; instead, OpenAI promises to improve the Git client upstream.

What to do with this

If your agents create many branches and parallel pushes, the bottleneck may soon shift from the model to Git hosting. Determine which processes truly require a globally consistent repository snapshot and which only need ordering within a single branch. It is also worth measuring push-to-fetch latency: this metric shows how long an agent waits before another agent or CI sees its commit. And watch for upstream Git improvements: they will probably reach everyone.