What Google Investors Should Really Be Cheering
What Google Investors Should Really Be Cheering
Google's new office at St. John's Terminal in New York City. Photo via AP.
Last week’s rally in stock of Google parent Alphabet was attributed to the company’s introduction of a dividend, its big improvement in its operating profit margin and faster growth at Google Cloud. But investors might have been also responding to another piece of good news that got little attention: Google has managed to lower the cost of showing artificial intelligence–generated answers to queries 80% over the last year, according to Alphabet CEO Sundar Pichai.
Pichai’s revelation may have more significance than anything else coming out of the Google earnings figures. Its search engine accounts for half of the company’s revenue and a big portion of its profits. But when Microsoft in early 2023 added an AI chatbot and responses that summarized information from the web to its Bing search engine, investors fretted that Google would have to spend heavily to defend its market share, undermining its overall profits.
The Takeaway
• Google slashed cost of AI-generated search by 80%
• Success relieves worries AI would hurt search profit margins
• Google put AI veterans in charge of search unit
Pichai’s comments, combined with the fact that Google’s share of the search market has barely budged over the past year, suggests those worries were unfounded. “People questioned whether these things would be costly to serve, and we are very, very confident we can manage the cost of how to serve these queries,” Pichai said on the earnings call last week.
Google began rolling out AI-generated search results to a group of users who had opted in last May. Instead of showing links where users can find the answer to a question, the feature generates a response based on information from the web. (Users can still scroll down to find links or to click on sources the AI response cites.)
In March, Google began adding AI responses to all searches, although the company has said it expects only a small percentage of users to see the new results at first. Google has reason to move cautiously, given the high cost of running AI models. Last year, Alphabet board chair John Hennessy told Reuters the cost of providing responses with an AI chatbot could be 10 times greater than that of a traditional Google search.
Pichai said Google drove down costs with “hardware, engineering and technical breakthroughs,” a reference to its Gemini AI model and custom AI chips that helped it achieve the savings. A person who has worked on search at Google said the company experimented with using different-size AI models to answer different types of queries. A smaller version of Gemini, which would use less computing power, might answer a relatively simple query. More-sophisticated queries would be routed to Gemini Ultra, the company’s flagship model, which is more expensive to use.
In addition to achieving cost reductions, Google has also tried to make the feature more efficient by triggering it only when the company thinks it would be better than a traditional set of search results, this person said. For example, asking Google for the weather wouldn’t bring up an AI result, but a more-complex query, like how to change a tire, might.
Even an 80% cost reduction would still make the AI searches pricier than normal search results. Nonetheless, the big cost reduction could translate to billions of dollars in savings in the long run if Google decides to introduce its AI features more widely.
Indicating how much of a priority Google has put on AI, the company shook up its search unit in March, elevating AI veterans. It put Liz Reid, who had overseen the development of new generative AI features, in charge of different search teams. Google also elevated Cheenu Venkatachary, an executive with expertise in large language models, to the role of head of search quality and ranking, replacing longtime executive Pandu Nayak. Hema Budaraju, a senior director of product in the search unit, leads product work on Search Generative Experience.
In addition to staff in the search unit, employees from DeepMind, Google’s AI research unit, have played a significant role in improving AI-generated search results in recent months—for instance, making them more accurate, said the person who has worked at Google.
Revenue Questions
Even as Google has brought down the costs of AI-generated search, the question of how to make money from AI remains unanswered. One major question is how Google will integrate ads into the AI-generated results. A year ago, when it previewed a pilot of Search Generative Experience, Google outlined ways the AI search feature could integrate ads, but it’s unclear how far along those efforts are.
For some searches, Google shows traditional ads that sometimes run above AI-generated results. Type in “how to buy a bracelet” and the top results are a group of ads, followed a little way down by the AI-generated explanation.
Pichai revealed another positive sign that emerged from the initial use of AI-powered search: Google was noticing that people who used AI-powered search actually searched more. That’s a good indication for Google’s search ads business, as getting users to search more provides more opportunities to show ads.
Internally, staff have discussed the impact that the AI-generated responses will have on display advertising on websites, according to two people who have worked at Google. Because these responses summarize an answer more succinctly than traditional search—which surfaces links to articles that might provide the answer—someone searching for information may not need to click through to another page to get more details. That could hurt revenue for some website publishers who rely on traffic from the search engine for display ad impressions.
Google has been under pressure for more than a year to strike a balance between cutting costs while rolling out new products that leverage advanced AI. The improvements in efficiency for Search Generative Experience have come alongside a massive increase in capital expenditures. On Thursday, Google said capex had doubled to $12 billion compared to a year ago, mostly because of investments in technical infrastructure for AI.
Still, Google is also hoping its investments in training Gemini will lead to new revenue streams—for example, via subscriptions to an advanced version of its ChatGPT competitor and selling access to the model through Google Cloud.