TheChek News Flow

“Nvidia will not ship GPUs unless the customer can certify that they have data center capacity in which to place those GPUs,” said Raul Martynek, CEO of DataBank, a data center provider whose clients include cloud providers.

Huang has become a business rock star and the chief cheerleader of an AI boom that has propelled his microchip firm’s once-in-a-generation growth and profits, lifting its value to the same $3 trillion level enjoyed by both Microsoft and Apple. But behind the glamour and well-deserved victory laps, Huang and his colleagues have also focused on countering the next threat to the business—the likelihood that demand for Nvidia’s chips will eventually slow down.

To guard against that possibility, Nvidia has begun selling more software to AI developers and a year ago even set up its own server rental business, DGX Cloud. That move put it directly into competition with its biggest customers—cloud providers such as Microsoft and AWS. Bizarrely enough, DGX Cloud operates on clusters of Nvidia-powered servers that it leases from those cloud providers. Nvidia then rents the servers to its own customers at a higher cost, promising them better computing performance.

The move has created tensions within the industry. AWS, the biggest cloud provider, initially resisted letting Nvidia carve out its own rival business within AWS data centers. But after all of AWS’ smaller rivals agreed to Nvidia’s terms, AWS relented, saying it would offer DGX cloud with a newer Nvidia AI chip that other cloud providers didn't have yet. AWS also may have been concerned about upsetting a critical supplier at a time when its chips were hard to come by.

Last fall, Nvidia even considered leasing its own data centers for DGX Cloud, according to a person who was involved in those discussions. Such a move would have cut out the cloud providers entirely. Nvidia also recently hired a senior Meta Platforms executive, Alexis Black Bjorlin, to run the cloud business. It isn’t clear whether Nvidia plans to move ahead with its own data centers for DGX Cloud.

As Nvidia takes these steps, its salespeople are going to great lengths to understand what customers are doing with Nvidia’s chips. Among the questions Nvidia’s salespeople are asking cloud providers lately is who their customers are and what kind of commitments they are signing to rent those servers. The answers could help Nvidia plan ahead for sales and could also help it learn about prospective customers for its own cloud server rental business.

While Huang manages these various efforts, he remains conscious of one factor that could affect sales: the fact that big tech companies buying his chips are making a large investment with an uncertain return. Microsoft, Meta, Elon Musk’s xAI and others are using chips to train experimental new AI models, which do not immediately generate revenue. When Meta CEO Mark Zuckerberg acknowledged the uncertainty about revenue in late April, the company’s stock dropped sharply as investors expressed their dissatisfaction with the situation. It’s conceivable investor pressure could prompt some of these companies to pull back on their chip purchases.

At the same time, Microsoft, AWS and other cloud providers have experienced a resurgence of demand for traditional computing workloads, not just AI, so they can’t afford to expand their data centers only to accommodate Nvidia chips, according to two people who work for one of the major cloud providers, as well as executives of several data center operators.

With these issues in mind, Huang has been carefully managing how Nvidia allocates chips so no one company amasses too many of them, colleagues and customers say. He’s also tried to influence how customers assemble GPUs in their data centers, pushing them to follow server-rack designs Nvidia thinks will lead to better computing performance.

But following Nvidia’s suggestions would make it harder for customers to shift to competing chips if they wanted to do so later, according to people at Nvidia and with some of its customers. This has led to periodic stand-offs with a key customer, Microsoft, over how the cloud provider planned to install Nvidia’s forthcoming chips, said a person who has been involved in the talks.

Nvidia also is looking to squeeze as much revenue as it can from the cables, racks and other hardware that connects the servers housing its chips, possibly at the expense of server manufacturers such as Dell that have long made servers with Nvidia chips.

Nvidia’s revenue from selling software and cloud services is tiny compared to its core server chip business. But in August last year, Chief Financial Officer Colette Kress said the new businesses already were on pace to generate hundreds of millions of dollars annually, and three months later she said that they would finish 2023 on pace to generate more than $1 billion annually, meaning they now contribute about 1% of Nvidia’s total revenue. In contrast, its core server chip business generated $47.5 billion last year.

In May, Nvidia disclosed that it had committed to spend nearly $9 billion on renting cloud servers from its top customers, mainly for internal research and development but also to boost its cloud server rental business. Some customers and former employees believe that business could eventually insulate Nvidia from an inevitable chip downturn and make it more difficult for customers who are renting its servers to pursue alternative chips.

The cloud and software products are “underappreciated by the analysts and technology community” as a business that could generate tens of billions of dollars a year in revenue, said Sasha Ostojic, a former Nvidia executive who is now a partner at venture firm Playground Global. “Nvidia has all of the leverage” it needs to grow services that complement its chips, he said.

On top of being a potentially lucrative revenue stream, DGX Cloud has become a way for Nvidia to help some of its customers transition to its newer-generation chips. For example, software maker ServiceNow, which has historically bought Nvidia servers for its own data centers, now also rents these servers from Nvidia directly.

Last year, when demand for GPUs was “so high, I said, ‘Hey, Jensen, I may need more [servers],’” said C.J. Desai, ServiceNow’s president. Huang told him: “That’s totally fine, but you should be able to burst the capacity into DGX Cloud.”

Desai said ServiceNow’s AI ambitions are “very dependent” on Nvidia’s products, including software introduced earlier this year that helps companies run AI efficiently in their apps. Desai declined to discuss how much his company is spending on the products, but said the amount has been growing and ServiceNow has turned down offers from rival cloud and chip firms because its engineers strongly prefer Nvidia’s “full stack.”

An Nvidia spokesperson declined to make Huang available for an interview for this article. “We prove our value to customers every day. Nvidia offers customers the lowest total cost of ownership, exceptional performance, and innovations at every layer of the stack—from chips to systems to software and algorithms,” the spokesperson said.

Huang’s moves reflect his paranoia as a founder who has survived multiple moments of near doom during the company’s history, including after it went public in 1999. The Christmastime meetings last year, for instance, could merely be part of the formula that has carried the company to great heights—turning its GPUs into the lifeblood of OpenAI, biotech and pharmaceutical firms, quant trading firms and scores of other AI developers.

“There’s no complacency at Nvidia,” said Jeff Herbst, a venture capitalist who spent two decades at Nvidia leading business development and acquisitions until 2021. “You wouldn’t really know whether times are good or times are bad from the tone or the tenor of the meetings.”

Nvidia got its start 31 years ago selling GPUs for PC gaming systems. Huang laid the foundation for Nvidia’s recent ascent in 2006 by launching Compute Unified Device Architecture, a programming language that taps into computing power provided by graphics chips. CUDA saved developers time by automating the process of building applications that harnessed the chips. In recent years CUDA has become a major factor in Nvidia’s sales: Millions of programmers don’t want to bother learning how to program with rival chips.

Nvidia sales began soaring in the 2010s, after AI developers such as Google began utilizing the chips to train large machine-learning models known as deep neural networks. Those AI models helped companies tailor websites and advertisements to individual customers, as well as boosting the accuracy of facial recognition and voice assistants’ ability to recognize people’s voices.

Huang has also had misses, including an audacious effort to develop software, not just chips, for self-driving cars, which involved hiring a substantial team of engineers. The plan didn’t work out. But the company got a boost in 2019 following its $7 billion acquisition of Mellanox Technologies, which gave it a stronger foothold in corporate data centers—locations where its AI chips have increasingly ended up.

“He wants to revolutionize health care, robotics, manufacturing, and in order to do that, he thinks, ‘What do I need to do now to enable that?’” said Umesh Padval, a venture capitalist who was on the board of Mellanox when Nvidia bought it.

OpenAI’s launch of ChatGPT, built using Nvidia chips in Microsoft data centers, sparked a nearly unprecedented boom for the chipmaker. Nearly every major technology firm and countless other developers scrambled for GPUs to develop their own conversational AI and models that generate images and video based on descriptions of what people want to see. Nvidia also has provided capital to scores of those developers, including Mistral, Cohere, Runway, Wayve, Figure and Perplexity. Nvidia’s market capitalization has risen eight times to $3.2 trillion since ChatGPT’s launch, and on June 5 Nvidia briefly overtook Apple as the world’s second most valuable public company.

At Computex, the tech industry conference in Taipei the first week of June, throngs of press chased Huang around town. In an indication of Huang’s stardom, a “Jensen Huang food map” illustrating eateries patronized by the black-leather-jacketed executive was hailed online as the hippest Taipei gourmet guide.

“He’s like Taylor Swift, but for tech,” Meta CEO Mark Zuckerberg said on Instagram after posting a photo of himself with Huang in March.

Though the 61-year-old Huang has soaked up the limelight, he has been managing tricky relationships with companies like Microsoft that are buying Nvidia’s chips while at the same time trying to lessen their reliance on those chips.

There is an inherent vulnerability in the business of selling chips on a one-off basis: As fast as sales have been rising, they could drop as demand inevitably cools off.

For Nvidia, a future without a steady new profit stream might not be pretty: As numerous commentators have pointed out, in 2000, Cisco Systems suddenly became the world’s most valuable company from selling routers at the height of the dot-com bubble, when telecom built new data centers, only to watch those centers go unused as internet-based revenue failed to materialize the way technology executives and investors thought it would. Cisco hasn’t recovered from the sales dropoff it experienced as its hardware became a widely available commodity.

Privately, Huang has told colleagues Nvidia must make sure it doesn’t end up like companies such as Cisco or Sun Microsystems, referring to their quick rise and eventual fall. Sun became a juggernaut in server and computer hardware in the 1990s, but after the bubble burst, the company didn’t capitalize on the burgeoning software market, which Microsoft and others captured. “He tries to remind people not to get ‘Sunned,’” said one Nvidia employee who has heard him say it.

Over the past few months, Nvidia has launched several software products it hopes will diversify its business from hardware. On an earnings call in February, Huang described the business, Nvidia AI Enterprise, as an “operating system for artificial intelligence” that customers would use to train and run AI. Nvidia charges $4,500 per GPU per year for access to the software. “My guess is that every enterprise in the world, every software enterprise company…will run on Nvidia AI Enterprise,” Huang said. “And so this is going to likely be a very significant business over time.” Nvidia has said design software maker Adobe and cybersecurity firm CrowdStrike are among the customers for the system.

“He is not selling chips—he is selling GPUs, software and systems for modern data centers,” Padval said. “People think it is just a GPU company, but it is much more than that.”

Even if the software business doesn’t grow as quickly as Nvidia hopes, it could drive more loyalty to the company’s chips and shield its core business from cheaper competitors down the line—including the AI chips each of the major cloud providers is trying to sell as an alternative to Nvidia’s.

“If you’re Nvidia, what you really want is this very powerful flywheel of better software services that keep customers in your orbit,” said Aaron Levie, CEO of storage firm Box, which rents Nvidia GPUs and resells Nvidia’s software for running AI to its own customers. “I think that [Huang]’s running that playbook quite well.”

In yet another effort to generate more hardware revenue, Nvidia is trying to have more influence over how its largest customers buy and install its GPUs. Typically, large cloud providers build their own customer server racks, which they use across their global data centers and for various kinds of chips. But when Nvidia approached customers about its next flagship chip, the GB200, it tried to convince them to buy the rack exactly as it had designed it, according to several people who have been involved in the talks.

Microsoft and Nvidia feuded over the issue for several weeks this year. Andrew Bell, a vice president at Nvidia, asked counterparts at Microsoft to buy a server rack design that was a few inches different in measurement from the racks Microsoft uses in its data centers, according to someone who was involved in the talks. Such a change would hinder Microsoft’s ability to easily switch between different AI chips. Bell said customers who agreed to buy Nvidia’s server rack design could be first in line to receive its new chips, but Microsoft executives demurred.

Microsoft executives had already felt pressured to buy Nvidia’s networking cables, which connect Nvidia servers to each other, because they believed Nvidia would prioritize such purchases over those that involved only its GPUs, according to a former Microsoft executive who was involved in it. Nvidia networking chips and cables accounted for roughly a third of the money Microsoft was spending on Nvidia products as of early 2023, this person said.

The dispute over the server rack design eventually reached the desks of Microsoft Chief Technology Officer Kevin Scott and CEO Satya Nadella, said one of the people who was involved. In the end, Nvidia backed down and agreed to let Microsoft design its own custom racks for the GB200 chips. (Google and AWS are also expected to produce custom racks for GB200s, according to two Nvidia employees.)

But Nvidia may still end up making more money from the GB200 racks smaller customers buy. In a change from prior years, Nvidia this year plans to design and procure the materials for servers and racks that hold GB200 chips before handing them off to the server manufacturers that build and ultimately sell them, according to two Nvidia employees who have been involved in the plan. The change would hurt server makers Dell, HPE and Supermicro, which have predominantly designed such hardware in the past. Those companies generate a higher margin from the racks they design than from using Nvidia’s designs, according to a manager at a major server assembler. (Spokespeople and executives from Dell and Supermicro didn’t respond to written requests for comment. A spokesperson from HPE declined to comment. )

As he tries to build new software businesses, Huang is attempting to maximize the growth of hardware sales while ensuring no single customer gets leverage over Nvidia. That might explain last week’s announcement that Oracle, a relatively small cloud server provider, would obtain a large number of Nvidia chips and start renting them out to Microsoft and OpenAI by early next year. Microsoft would have preferred to purchase those chips directly, though one person with knowledge of the deal said it might have struggled to find space and power for that many new chips.

Huang also has maintained a special relationship with CoreWeave, another small cloud provider whose revenue last year soared to hundreds of millions of dollars, up from $25 million in 2022, after it got generous allotments of Nvidia GPUs at the expense of bigger cloud providers. Microsoft also had to rent GPU server capacity from CoreWeave to meet its needs.

To be sure, there’s no imminent risk to Nvidia’s hardware sales. The company has generated $40 billion in free cash flow in the nine months that ended in April thanks to its unrivaled technology, and it says demand for the next flagship chips, due out later this year, are through the roof.

Nvidia’s Kress said in May that purchases of GPUs by governments or government-backed companies would lead to sales in the “high-single-digit billions” this year, implying that the customer segment had plenty of steam left.

Still, the price to rent Nvidia’s H100 chips in the cloud has fallen nearly 30% in the past year, and that could indicate a slowdown is coming, said Dylan Patel, chief analyst at chip research firm SemiAnalysis, which also monitors cloud provider prices. Compared to last year, companies can also more easily rent the chips without committing to multiyear deals with cloud providers, he said.

Another issue: While the biggest technology companies have the expertise to handle Nvidia’s most advanced chips, which require special cooling and other conditions, many startups, multinationals, governments and academic buyers do not, according to some of these customers as well as the companies that do business with them. So as Nvidia flagship H100 chips have become easier to find this year following a prolonged shortage last year, less-sophisticated buyers haven’t been using them the way they are intended—essentially, it’s like they’re driving a Ferrari on slow city streets.

That means as Nvidia chips become more expensive, some of these customers may not want to upgrade. “We’ve found that oftentimes you don’t need the latest generation of Nvidia GPU to get great performance,” said Matthew Prince, CEO of Cloudflare, which operates a cloud service.