NVIDIA Lays Out Its Plan for AI Computing at COMPUTEX 2024

NVIDIA CEO Jensen Huang gave a keynote presentation at COMPUTEX 2024 in Taipei outlining how the company plans to build the computing infrastructure that runs AI systems. NVIDIA makes the chips that power artificial intelligence, and Huang's remarks signaled the company's strategy to do more than just sell hardware—to shape the entire ecosystem around it.

His presentation focused on what comes after the initial wave of large language models, the AI systems that can write and reason with text. The industry is now grappling with a harder problem: how to run these AI systems reliably and cheaply once they're deployed in the real world, serving millions of users.

What NVIDIA Is Pushing

The keynote emphasized practical, business-focused AI infrastructure. Marc Hamilton, an NVIDIA executive, gave a separate presentation titled "Infra Build Train Go," which focused on the concrete engineering challenges that companies face when deploying AI at scale.

The presentations, delivered during COMPUTEX 2024's main exhibition in early June, positioned NVIDIA not as a chip vendor but as a company thinking through the entire pipeline: building AI systems, training them on data, and then putting them into production.

NVIDIA also held a separate AI Summit at the same conference, which gave them space to discuss how different companies and partners could work together to deploy AI.

Why Taiwan Matters

The timing and location are worth noting. Taiwan is the world's manufacturing hub for semiconductors and computer chips. By announcing their AI strategy at a major conference in Taiwan, NVIDIA was signaling to manufacturers, suppliers, and customers that AI infrastructure requires coordination across many companies and regions.

The choice also underscores the geopolitical importance of AI computing. NVIDIA relies on Taiwan and other Asian partners to manufacture and distribute its chips, even as the company expands its software services. For companies buying AI infrastructure, this regional expertise and supply chain access matters for long-term planning.

The Shift from Pure Hardware

In this author's view, what NVIDIA is doing here parallels a pattern we have seen before. About a decade ago, NVIDIA moved beyond making graphics chips and created CUDA, a software platform that let programmers use NVIDIA chips for all kinds of computing tasks, not just rendering video. Now the company is again trying to shape the entire stack—the chips, the software, the tools, and the services—rather than just selling silicon.

This works when one company can convince the rest of the industry to build around their foundation. It is harder than it sounds, and the AI infrastructure market is still more fragmented than, say, the smartphone or personal computer markets of earlier eras. But that is NVIDIA's bet.

What Companies Actually Need

Many organizations have successfully built early AI projects—proof-of-concept systems that show promise. But moving from an experiment to a production system that runs reliably day after day turns out to be much harder. Companies need tools to manage different versions of AI models, to optimize how quickly the system responds to requests, and to keep everything running smoothly.

Hamilton's "Infra Build Train Go" framework appears designed to address exactly that gap. Enterprise customers have told NVIDIA, in effect: we have trained an AI system, now what do we do with it. Enterprise deployments are not like consumer products, where you can update and change things quickly. They require predictable, reliable performance and careful management.

Where the Real Work Is

The presentations suggest that NVIDIA sees an opportunity in the harder operational problems, not just in raw computing power. As AI systems move from research labs into companies, the bottlenecks shift. It is no longer just about how fast your chips are. It is about optimizing how the AI system responds to requests, managing memory efficiently, and ensuring that the whole infrastructure works together smoothly.

This reflects a maturation in how companies are using AI. Early on, organizations built systems to train large AI models, and that required enormous amounts of computing power. Now they are running those trained models millions of times a day, answering customer questions or making predictions. Running a trained model is a different problem than training it, with different technical requirements.

NVIDIA's strategy suggests the company believes that companies making long-term AI infrastructure commitments will want to buy more than just chips. They will want the software, the tools, and the operational support that comes with them.