Google launches TPU 8i Max and 8tx: chips for agents and sensitive data

If you have ever run an agent that spends an hour waiting for a response from a user or an external API, you know the pain point: the accelerator remains occupied without doing useful work. Then there is long context, which runs up against memory limits and the speed of KV-cache transfers between nodes. In its latest announcement, Google aims to address both problems at once and adds a third capability: processing sensitive data without having to trust the host.

The hardware

The eighth-generation TPU lineup now consists of four systems. The two additions are TPU 8i Max and TPU 8tx.

TPU 8i Max is essentially two TPU 8i chips joined together. Google says it delivers twice the computing power, twice the SRAM and twice the HBM bandwidth. Its target workloads are massive context windows and complex agent tasks. Memory capacity and bandwidth matter more here than raw teraflops: with long contexts, most of the time goes into reading the KV cache rather than matrix multiplication.

TPU 8tx is designed for confidential computing. Hardware protects data during processing, covering both inference and fine-tuning. For banking, healthcare and legal products, this could potentially remove the main objection to cloud models: “We cannot send this kind of data outside our organization.”

Software around the chips

This is where the most practical part of the announcement lies.

Agent Compute Engine offloads the state of idle agent sessions and frees up accelerators. Imagine a customer support agent that asks a question and waits a full day for the answer. Previously, its context occupied memory on an expensive chip; now its state is offloaded, allowing the chip to serve other sessions.

TPU Sync reduces KV-cache transfer latency by more than 90%, according to Google. This matters when the prefill and decode stages run on separate nodes: the faster the cache moves, the shorter the wait for the first token.

XLA Serving Engine now has an open-source C++ engine for serving models on TPUs, GPUs and CPUs. For teams concerned about vendor lock-in, this is a significant signal: the same serving layer can run on different hardware.

GKE Dynamic Slicing speeds up job startup by up to 5 times and recovery from failures by up to 4.5 times. For long training and fine-tuning runs, that translates directly into cost savings through less cluster downtime after a node failure.

Gemini designs chips

Another detail: Google uses Gemini and TPU computing resources to create component placement layouts. The company says this takes hours instead of weeks. This is Google's own claim, with no independent confirmation, but the trend is clear: AI is starting to accelerate the development of its own hardware.

Limitations

All figures, including the 90% reduction for KV-cache transfers and the 5-fold GKE speedup, come from Google materials and have not been independently verified. The announcement provides no pricing, availability timelines or regions. There is no comparison with Nvidia either, so it is too early to judge any advantage in cost per token. Confidential computing on 8tx warrants separate scrutiny: exactly which threats it addresses, how attestation works and how performance changes when protection is enabled.

What to do in practice

If you are building agent products with long sessions, calculate how much time your accelerators spend idle while waiting. That is where Agent Compute Engine will make a difference. If you work with regulated data, consider 8tx as a point to raise in discussions with your security team. If vendor lock-in concerns you, explore the open-source XLA Serving Engine: you can try it regardless of your choice of chip. Product teams should wait for pricing and benchmarks before rebuilding their infrastructure.