NVIDIA's Plan for the Next Era of AI: What Happened at COMPUTEX 2024

NVIDIA CEO Jensen Huang used his keynote at COMPUTEX 2024 in Taipei to lay out how the company plans to shape the future of AI infrastructure — not just the computer chips themselves, but the entire ecosystem around them.

The presentation came as companies worldwide struggle with a practical problem: they have built and trained AI models, but now they need to run them reliably in production environments. That shift — from experiments to everyday business use — is where NVIDIA is focusing its attention.

From Chips Alone to the Whole Stack

Marc Hamilton, NVIDIA's vice president of solutions architecture and engineering, gave a session titled "Infra Build Train Go" at the TAITRA forum on June 5. His message was direct: enterprise customers need more than just faster hardware. They need tools and frameworks that take them from initial training all the way through to running their AI systems in the real world.

This is a shift in how NVIDIA presents itself. The company built its fortune selling GPUs — specialized chips that excel at handling AI workloads — but now it is positioning itself as the architect of the whole pipeline. An AI summit held at the Grand Hilai Taipei on the same day gave technical teams deeper dives into how that pipeline could work across different companies and setups.

A Pattern We've Seen Before

NVIDIA did something similar about a decade ago when it moved from making graphics chips to creating CUDA, a platform that lets programmers use its hardware for any kind of computation, not just graphics. The company succeeded partly because it didn't just sell chips — it built software, tools, and partnerships that made those chips indispensable.

The current push follows the same playbook. Rather than wait for customers to build their own solutions, NVIDIA is defining the standards and tools for the whole AI production process. There is historical precedent for this approach: Microsoft dominated the PC era by controlling not just hardware but the software ecosystem, and Apple created a tightly integrated stack in mobile. The AI infrastructure market is more fragmented than those earlier eras were, but NVIDIA's strategy suggests the company believes it can shape how the industry consolidates.

Why Taiwan and Why Now

COMPUTEX is traditionally where hardware makers and the companies that assemble systems meet. Taiwan is the manufacturing and design center for the world's most advanced chips. For NVIDIA to make infrastructure announcements there sends a message: this is not just about America or one company working alone, but about coordinating across global supply chains and multiple partners.

The timing also reflects what is happening in boardrooms around the world. Companies have spent the last year running proof-of-concept AI projects. Many succeeded. But when teams tried to move those experiments into production — to run AI systems on real customer data, reliably and cheaply — they hit bottlenecks that pure computing power alone cannot solve. Model versioning, making sure inference (running a trained model on new data) stays fast, and managing all the infrastructure that supports it became the real challenge.

What Enterprises Actually Need

Enterprise customers care about different things than researchers do. A researcher can accept that an experiment will take two weeks to run and sometimes produce inconsistent results. A company running AI in production needs the system to work the same way every time, to give results quickly, and to integrate cleanly with everything else it already does.

NVIDIA's "Infra Build Train Go" framework addresses that gap. It is designed to help organizations move from "we ran an AI experiment in a lab" to "AI is part of how we run our business." That requires not just powerful chips but orchestration software, monitoring tools, and standardized ways to move trained models from development environments into live systems.

The Real Challenge: Beyond Raw Speed

For a long time, AI infrastructure improvements came down to making chips faster. But as AI moves into production use, other bottlenecks appear. The memory (how much data the system can hold at once), the network connecting different machines, and how efficiently the software uses all that hardware matter just as much as raw speed.

This is especially true for inference, which is running a trained model on new data to get predictions. Inference workloads look very different from training workloads — they need to be fast and predictable rather than powerful and flexible. NVIDIA's emphasis on this area suggests the company sees inference as the next big opportunity, since most AI systems in production spend their time doing inference, not training.

By tying its hardware roadmap to these operational realities, NVIDIA is betting that the companies best positioned to win in AI infrastructure will be those that solve the entire problem — silicon, software, and everything in between — not just one piece of it.

The Broader Context

NVIDIA's shift from "we sell the best GPU" to "we run your entire AI operation" mirrors patterns we have seen in previous computing eras. But unlike those earlier eras, where one company sometimes achieved near-monopoly control, the AI infrastructure market has many competitors and many different approaches. Whether NVIDIA can again shape an entire ecosystem the way it did with CUDA remains an open question. What is clear is that the company is trying, and that companies evaluating AI infrastructure should watch whether NVIDIA's approach becomes a standard or whether competitors offer meaningful alternatives.