At GTC 2026, NVIDIA shifts focus from chips to token factories and the Agent-as-a-Service era

At GTC 2026, NVIDIA argued that the next stage of the AI market will be built not around individual GPUs but around inference factories, where tokens and agent actions become the main product. The keynote's central thesis: the company no longer sells just accelerators, but a complete infrastructure for industrial-scale production of AI output, from CUDA libraries to server racks, networking and the enterprise software layer.

Twenty years of CUDA as the foundation

The starting point for this pivot was twenty years of CUDA. It was the bet on a software platform that once turned NVIDIA graphics cards from niche hardware into a general-purpose compute tool for machine learning. At GTC, that path was presented as a steady evolution: first an ecosystem of libraries and frameworks, then DGX systems, and now ready-made modular building blocks for large AI clusters. The logic is simple: even the most powerful chip means little without software, optimizations and the ability to deploy applied workloads quickly in production.

From SaaS to Agent-as-a-Service

That leads to NVIDIA's second thesis: the market is moving from SaaS to Agent-as-a-Service. Where companies once paid for access to a tool and an employee produced the result, businesses are now starting to pay for a completed AI action. An agent should not merely generate text but close out a task: handle a request, run an analysis, prepare a document, make a decision within set rules. So the yardstick of efficiency becomes not abstract performance in FLOPS but the cost of a useful token and the final price of a meaningful action. In this logic, inference turns into an economy of its own, and data centers into production sites for intellectual work.

The Token Factory concept

This is where NVIDIA pushes its Token Factory concept. The company suggests viewing modern AI data centers not as places to store data but as factories where electricity and infrastructure go in and a stream of tokens for applications, assistants and autonomous agents comes out. At GTC, an estimate was given that by 2027 global spending on building and upgrading such capacity could approach $1 trillion. Demand is fueled not only by enterprise AI but also by the growth of open models, which have come close to the frontier in quality and make it easier for more companies to launch their own services.

The Vera Rubin architecture

The hardware backbone of this strategy is the Vera Rubin architecture. NVIDIA describes it not as another performance gain over the previous generation but as an attempt to repackage the entire stack for inference. What matters is no longer a single card or even a single server, but the whole rack: compute, CPU, memory, storage, networking, security and optical links between modules.

This approach is meant to raise throughput and responsiveness at the same time without blowing up electricity costs. A separate emphasis is on modularity: configurations can be assembled for different workload types, from mass-market fast responses to expensive real-time reasoning. That also creates a new market segmentation: cheap answers for broad consumption and premium inference for complex agentic scenarios.

Another important signal from GTC is that AI agents are increasingly treated as part of corporate infrastructure rather than an experimental layer on top of chats. So alongside the hardware, NVIDIA is promoting software reference architectures for deploying agents in large businesses. The point is for an agent to work within security policies, have access only to permitted interfaces and fit predictably into a company's existing IT landscape.

For enterprises this may even matter more than the chips themselves: without control, audit and manageability, no autonomy will reach production.

The bottom line

The main takeaway from GTC 2026 is that NVIDIA is trying to claim not only the accelerator market but also the position of base supplier of the inference economy. Where competition once centered on transistor counts and leadership in model training, the center of gravity is now shifting to the cost of a useful action, the resilience of AI services and the speed of deploying agentic systems. For the market, this means moving from talk of the "most powerful GPU" to the question of who can produce intelligence as a service more cheaply and reliably.

In brief:

• At GTC 2026, NVIDIA shifted focus from individual GPUs to inference factories, where tokens and agent actions become the product.
• The market is moving from SaaS to Agent-as-a-Service; the yardstick is the cost of a useful token. Spending on such capacity could approach $1 trillion by 2027.
• The Vera Rubin architecture repackages the whole stack for inference: the entire rack, with an emphasis on modularity for different workloads.