Google takes aim at Nvidia with new AI inference chips

Google is taking on Nvidia. The company is developing a new generation of in-house AI chips built for inference: the final stage of a neural network's work, when a trained model handles user requests in real time. Inference chips differ fundamentally from the ones used to train models.

Training a large language model means months of computation on thousands of GPUs, a one-time capital expense. Inference is a constant load: every time a user sends a request to Gemini, ChatGPT, or any other AI service, a chip runs inference. As AI apps grow to hundreds of millions of users, inference becomes the biggest line item in tech companies' budgets.

By some estimates, inference costs will exceed training costs by three to four times by 2027. Google has been building its own silicon strategy for a long time. It has been developing TPUs (Tensor Processing Units) since 2015, long before AI chips made front-page news.

Until now, these processors have mostly been used inside Google Cloud and to train Gemini models. Now the company plans to build hardware specifically for inference, with higher throughput and lower power consumption per request. For Nvidia, whose H100 and H200 chips have become the standard in data centers worldwide, this is a direct challenge.

Google is one of Nvidia's largest customers, and shifting even part of its workload to in-house hardware would mean significant losses for the Santa Clara company. Meanwhile, another development is brewing in the AI chip sector: Cerebras Systems has announced plans to go public. The company is known for its flagship Wafer-Scale Engine, essentially an entire silicon wafer that operates as a single processor.

This architecture eliminates the latency of moving data between separate chips and speeds up processing of large models. Cerebras had tried to go public before but withdrew its filing. Its return to the IPO track sends a signal to the market: even as some AI investors cool off, alternative chip architectures are still seen as a promising asset.

The company pitches its products as especially efficient for running open models inside closed enterprise environments, a segment growing quickly amid security and data sovereignty requirements. The week's third story comes from space. Blue Origin successfully launched its New Glenn rocket and landed its reusable booster, an important technical milestone toward cheaper launches.

However, the payload, an AST SpaceMobile satellite, failed to reach its planned orbit. AST SpaceMobile is building a global broadband network that works directly with smartphones, without special terminals. The orbital failure hit the stock immediately: the company's shares fell.

The market once again showed that in space, technical success and commercial results are two different things. Taken together, these three stories sketch a portrait of the tech economy in the mid-2020s. The AI chip race is moving beyond a duopoly: alongside Nvidia and AMD, tech giants with their own silicon and specialized startups like Cerebras are joining the fight over infrastructure.

The question of who will control the physical infrastructure of the AI era remains open.