Google must double AI serving capacity every 6 months to meet demand, AI infrastructure boss tells employees
- At a recent all-hands meeting, Google’s head of AI infrastructure, Amin Vahdat, said the company has to race to build out compute capacity in order to meet demand.
- Vahdat said Google bolsters its AI infrastructure capabilities with more efficient models and through its custom silicon.
- In a presentation, Vahdat included a slide that said, “Now we must double every 6 months.... the next 1000x in 4-5 years”
- Google CEO Sundar Pichai spoke at the meeting and addressed employee questions about the potential of an AI bubble bursting and capital expenditures.
Google ’s
AI infrastructure boss told employees that the company has to double its serving capacity every six months in order to meet demand for artificial intelligence services.
At an all-hands meeting on Nov. 6, Amin Vahdat, a vice president at Google Cloud, gave a presentation, viewed by CNBC, titled “AI Infrastructure,” which included a slide on “AI compute demand.” The slide said, “Now we must double every 6 months.... the next 1000x in 4-5 years.”
“The competition in AI infrastructure is the most critical and also the most expensive part of the AI race,” Vahdat said at the meeting, where Alphabet CEO Sundar Pichai and CFO Anat Ashkenazi also took questions from employees.
The presentation was delivered a week after Alphabet reported better-than-expected third-quarter results and raised its capital expenditures forecast for the second time this year, to a range of $91 billion to $93 billion, followed by a “significant increase” in 2026. Hyperscaler peers Microsoft
, Amazon
and Meta
also boosted their capex guidance, and the four companies now expect to collectively spend more than $380 billion this year.
Google’s “job is of course to build this infrastructure but it’s not to outspend the competition, necessarily,” Vahdat said. “We’re going to spend a lot,” he said, adding that the real goal is to provide infrastructure that is far “more reliable, more performant and more scalable than what’s available anywhere else.”
In addition to infrastructure build-outs, Vahdat said Google bolsters capacity with more efficient models and through its custom silicon. Last week, Google announced the public launch of its seventh generation Tensor Processing Unit called Ironwood, which the company says is nearly 30 times more power efficient than its first Cloud TPU from 2018.
Vahdat said the company has a big advantage with DeepMind, which has research on what AI models can look like in future years.
Google needs to “be able to deliver 1,000 times more capability, compute, storage networking for essentially the same cost and increasingly, the same power, the same energy level,” Vahdat said. “It won’t be easy but through collaboration and co-design, we’re going to get there.”
Pichai told employees at the meeting that 2026 will be “intense,” citing AI competition and the pressure to meet cloud and compute demand.
He also answered a question about a potential AI bubble, a topic that’s gained resonance across Silicon Valley and Wall Street of late as investors have grown skeptical about whether the trillions of dollars in anticipated spend in the coming years is justified.
The employee question that he read aloud asked, “Amid significant Al investments and market talk of a potential Al bubble burst, how are we thinking about ensuring long-term sustainability and profitability if the Al market doesn’t mature as expected?”
Pichai acknowledged the concerns.
“It’s a great question. It’s been definitely in the zeitgeist, people are talking about it,” Pichai said.
He then reiterated a point he’s made in the past about the risks of not investing aggressively enough, and highlighted Google’s cloud business, which just recorded 34% annual revenue growth to more than $15 billion in the quarter. Its backlog reached $155 billion.
“I think it’s always difficult during these moments because the risk of underinvesting is pretty high,” Pichai said. “I actually think for how extraordinary the cloud numbers were, those numbers would have been much better if we had more compute.”
He said the company follows a disciplined approach, pointing to the strength of the underlying businesses and company’s balance sheet.
“We are better positioned to withstand, you know, misses, than other companies,” Pichai said.
Market jitters
Looking ahead to next year, Pichai told employees, “there will be no doubt ups and downs.”
“It’s a very competitive moment so, you can’t rest on your laurels,” he said. “We have a lot of hard work ahead but again, I think we are well positioned through this moment.”
Google declined to comment.