Distributed AI Training Is Changing Inter-Data-Center Networks: 5 µs per Kilometer

Every kilometer of optical fiber adds about 5 microseconds of latency. For a link between two data centers 100 km apart, that is 0.5 ms one way and roughly 1 ms round trip. At 500 km, round-trip latency is already about 5 ms. Inside a single GPU hall, it is measured in single to tens of microseconds. The Next Platform calls this gap of two to three orders of magnitude the main difficulty of training across multiple sites. The piece was published as sponsored content, and the available excerpt contains no data from any specific vendor. What follows is therefore an analysis of the problem itself, based on physics and well-known engineering approaches.

Why one campus is no longer enough

Clusters of hundreds of thousands of accelerators need hundreds of megawatts, and sometimes gigawatts. Finding that much power at a single point on the grid is getting harder: substation connections take years, and sites run into limits on cooling and land. The logical move is to spread a cluster across several campuses within tens or hundreds of kilometers and make them work as one machine. That is where the problems begin, because training network architecture was built around microseconds.

Where the familiar architecture breaks down

Synchronous training requires GPUs to exchange gradients after every step. For a trillion-parameter model in bf16, that is about 2 TB of data per iteration (an estimate that ignores sharding and compression). A single 800 Gbit/s channel carries roughly 100 GB/s, so on one link that volume would take tens of seconds. Between sites, you therefore need tens or hundreds of parallel channels, and traffic has to be split so that the heavy exchanges stay inside the building.

The second problem is packet loss and congestion. Modern AI networks are sensitive to packet loss, and on a long link the switch buffer must hold all the data that is "in flight." The longer the link and the higher the speed, the deeper the buffering has to be. That explains the interest in network chips with large memory and in switches designed for inter-site links.

Key parameters of the problem:

• fiber latency: ~5 µs/km, ~1 ms round trip at 100 km;
• channel: 800G coherent optics, tens of parallel links per pair of sites;
• exchange volume: on the order of terabytes of gradients per step for models at the trillion-parameter scale;
• packet loss: critical, so deep buffering and congestion control are required;
• main reserve: reducing the number of synchronizations between sites.

What engineers are doing

The first path is hardware. 800G ZR/ZR+ coherent modules can carry a signal for hundreds of kilometers without separate transponders, and next-generation network chips take over buffering and traffic balancing between campuses. The second path is software. Pipeline parallelism can be laid across sites: activations pass between stages less often and in smaller volumes than gradients. Dense tensor-parallelism exchange stays inside the building. The third path is changing the algorithm itself. Methods such as DiLoCo synchronize weights once every few hundred steps rather than at every step, giving up some efficiency in exchange for tolerance to latency. Gradient compression and overlapping computation with communication help hide milliseconds behind GPU work.

Practical takeaways

For cluster operators, the question is now framed differently: how many milliseconds can be hidden behind computation. If sites are 50–100 km apart, the losses are moderate and can be concealed. Beyond that, at 500 km, the cost climbs quickly unless parallelism is rethought. A team planning distributed training should budget latency and inter-site traffic volume in advance, not just count GPUs. When choosing network equipment, buffer depth, support for 800G coherent optics, and congestion telemetry matter. One more risk is the sponsored nature of the source: compare vendor claims against your own measurements on real workloads.