TheChek News Flow

The most powerful artificial-intelligence systems need a network to work their wonders. That’s a plus for Broadcom, Marvell, and others.

It takes a network to unleash the powers of artificial intelligence, and companies that supply these networks are seeing their stocks soar.

AI systems have gotten smarter as they have scaled up from a few Nvidia processors to thousands of chips. The recently unveiled Colossus AI supercomputer of Elon Musk’s xAI has 100,000 processors. Million-chip clusters are on drawing boards.

Nvidia processors are the most valuable parts of these factory-size AI systems, and that’s why Nvidia is one of the world’s most valuable companies. To put 100,000 processors to work on a single AI computing task, however, you need network technology that is as advanced as the processors themselves.

The basic computing element in AI isn’t a processor, but a data center, notes Nvidia networking chief Gilad Shainer. So, a growing portion of the billions spent on AI data centers will go to the suppliers of network chips, lasers, and switches that integrate thousands of processors into a single computer. Nvidia supplies that network infrastructure itself, but so do Broadcom, Cisco Systems, Arista Networks, Marvell Technology, and others.

AI can’t advance without advanced networks, says Shainer. “The network is the most important element because it determines the way the data center will behave.”

Shares of Broadcom leapt 40% in December after the networking company detailed its expanding opportunities in AI on its earnings call. Network chips now account for just 5% to 10% of all AI chip spending, said Broadcom CEO Hock Tan. But as the size of AI systems hits 500,000 or a million processors, Tan expects that networking will become 15% to 20% of a data center’s chip budget. A data center with a million or more processors will cost $100 billion to build.

Shares of Arista and Marvell also enjoyed big moves in 2024, as investors woke up to their opportunities in AI. None of the AI networkers remains a cheap stock, but they will be important ones in the AI industrial revolution.

The firms building the biggest AI clusters are called hyperscalers, and they are led by Alphabet’s Google, Amazon.com, Facebook-parent Meta Platforms, and Microsoft. Not far behind are Oracle, xAI, Alibaba Group Holding, and ByteDance. Capital spending by the hyperscalers rose more than 50% in 2024’s third quarter, to an annualized level above $200 billion. Goldman Sachs estimates that AI spending will rise another 35% to 40% in 2025.

AI architectures began scaling in recent years for two reasons. Processor chips from the likes of Intel neared the end of speed gains made possible by shrinking a chip’s transistors. Then, computer scientists at companies such as Google and OpenAI built AI models that performed amazing feats by finding connections within large volumes of training material. As the components of these “large language models” grew to millions, billions, and then trillions, they began translating languages, doing college homework, handling customer support, and designing cancer drugs.

But training an AI model is a huge task, as it calculates across billions of data points, rolls those results into new calculations, then repeats. Even with Nvidia accelerator chips to speed up those calculations, the workload has to be distributed across thousands of Nvidia processors and run for weeks.

To keep up with the distributed computing challenge, AI data centers have two networks. The “front end” network takes in data and communicates with the system’s users—like the networks of every enterprise data center or cloud-computing center.

What is new is a “back end” network that connects every AI processor and memory chip with every other processor. “It’s just a supercomputer made of many small processors,” says Ram Velaga, Broadcom’s chief of core switching. “All of these processors have to talk to each other as if they are directly connected.”

AI’s back-end networks need high bandwidth switches and network connections. Delays and congestion are expensive when each Nvidia compute node costs as much as $400,000. Idle processors waste money.

Back-end networks carry huge volumes of data. When thousands of processors are exchanging results, the data crossing one of these networks in a second can equal all the internet traffic in America.

That’s how Nvidia became one of today’s largest vendors of network gear. CEO Jensen Huang and his colleagues realized early on that AI workloads would exceed a single box. They started using InfiniBand, a network designed for scientific supercomputers, which was supplied by a company called Mellanox. InfiniBand became the standard for AI back-end networks.

Nvidia acquired Mellanox in 2020 for $6.9 billion. While most AI dollars still go to Nvidia accelerator chips, back-end networks are important enough that Nvidia has large networking sales. In 2024’s September quarter, those network sales grew 20%, to $3.1 billion.

Yet there’s a challenge to InfiniBand’s lock on AI networks and, perhaps, to Nvidia’s. It comes from Ethernet, the network standard that’s used for everything outside of AI back-ends, including the internet.

Ethernet lacks InfiniBand’s tools for memory and traffic management, but those are now being added in a version called Ultra Ethernet. Hyperscalers think Ethernet will outperform InfiniBand, as clusters scale to hundreds of thousands of processors. Another attraction is that Ethernet boasts many competing suppliers.

“All the largest guys—with an exception of Microsoft—have moved over to Ethernet,” says a network industry executive. “And even Microsoft has said that by summer of 2025, they’ll move over to Ethernet, too.”

Nvidia may be known for InfiniBand, but it sells Ethernet, as well. xAI uses Nvidia Ethernet products in its record-size Colossus system.

Ethernet back-end networks offer a big opportunity for Arista Networks, which builds switches using Broadcom chips. In the past two years, AI data centers became an important business for Arista.

AI provides sales to Arista switch rivals Cisco and Juniper Networks (soon to be a part of Hewlett Packard Enterprise), but those companies aren’t as established among hyperscalers. Analysts expect Arista to get more than $1 billion from AI sales in 2025 and predict that the total market for back-end switches could reach $15 billion in a few years. Three of the five big hyperscale operators are using Arista Ethernet switches in back-end networks, and the other two are testing them. Arista CEO Jayshree Ullal says that back-end network sales seem to pull along more orders for front-end gear, too.

The network chips used for AI switching are feats of engineering that rival AI processor chips. Cisco makes its own switching chips, but some 80% of the chips used in other Ethernet switches comes from Broadcom, with the rest supplied mainly by Marvell. These switch chips now move 51 terabits of data a second; it’s the same amount of data that a person would consume by watching videos for 200 days straight. In 2025, switching speeds will double.

The other important parts of a network are connections between computing nodes and cables. As the processor count rises, connections increase at a faster rate. A 25,000-processor cluster needs 75,000 interconnects. A million processors will need 10 million interconnects.

More of those connections will be fiberoptic, instead of copper.

AI processing chips exchange data at about 10 times the rate of a general-purpose processor chip. Copper has been the preferred conduit because it’s reliable and requires no extra power. At current network speeds, copper works well at lengths of up to five meters. So, hyperscalers have tried to “scale-up” within copper’s reach by packing as many processors as they can within each shelf, and rack of shelves.

But as networks speed up, copper’s reach shrinks. So, expanding clusters have to “scale-out” by linking their racks with optics. “Once you move beyond a few tens of thousand, or 100,000, processors, you cannot connect anything with copper—you have to connect them with optics,” Velaga says.

Back-end connections now run at 400 gigabits per second, which is equal to a day and half of video viewing. Broadcom’s Velaga says network speeds will rise to 800 gigabits in 2025, and 1.6 terabits in 2026.

Nvidia, Broadcom, and Marvell sell optical interface products, with Marvell enjoying a strong lead in 800-gigabit interconnects. A number of companies supply lasers for optical interconnects, including Coherent, Lumentum Holdings, Applied Optoelectronics, and Chinese vendors Innolight and Eoptolink. They will all battle for the AI data center over the next few years.

The opportunity for optical connections reaches beyond the AI data center. That’s because there isn’t enough power.

A 500,000-processor cluster needs at least 750 megawatts, enough to power 500,000 homes. When AI models scale to a million or more processors, they will require gigawatts of power and have to span more than one physical data center, says Velaga.

In September, Marvell, Lumentum, and Coherent demonstrated optical links for data centers as far apart as 300 miles. Nvidia’s next-generation networks will be ready to run a single AI workload across remote locations.

Some investors worry that AI performance will stop improving as processor counts scale. Nvidia’s Jensen Huang dismissed those concerns on his last conference call, saying that clusters of 100,000 processors or more will just be the table stakes with Nvidia’s next generation of chips.

And for that, Broadcom’s Velaga says he is grateful: “Jensen has created this massive opportunity for all of us.”