OpenAI Is Getting More Efficient at Running Its AI, Internal Financials Show
The Takeaway
- OpenAI’s compute margin for paying users jumped to 70% in October
- Company had declared a ‘Code Red’ to focus on server costs after release of new model from DeepSeek
- For total computing costs, Anthropic is expected to be more efficient
As OpenAI discusses raising an unprecedented investment round of up to $100 billion, it can crow about some key improvements to how it runs its business. While its total computing costs are still high as a percentage of revenue, it is wringing more revenue out of every dollar it spends to run the servers that power ChatGPT’s subscription business and selling access to models to corporate customers.
The company’s compute margin, its share of revenue after the cost of running AI models for paying users, has jumped to about 70% in October from about 52% at the end of last year and roughly 35% in January 2024, according to a person with knowledge of the company’s financials.
Rival Anthropic, in contrast, had a compute margin of about negative 90% last year, according to The Information’s analysis of its financials. It has been on track to improve that margin to about 53% by the end of this year and has projected a 68% compute margin next year in its most optimistic prediction, according to the analysis.
Overall, though, Anthropic’s projections show it will overtake OpenAI in total server efficiency, including running its AI models for nonpaying users and training its models. OpenAI has hundreds of millions of nonpaying chatbot users it will have to monetize through ads or shopping affiliate fees to make up for that gap. Anthropic has far fewer nonpaying users of its chatbot.
Anthropic this summer projected it will spend as much as $60 billion on overall computing costs, including to develop new AI, next year through 2028. These projections did not include recent deals for server rentals from Google and Microsoft. OpenAI forecasts spending $220 billion on servers through the same period.
OpenAI’s big outlays for servers reflect its leaders’ belief that a shortage of servers is the biggest challenge to the company’s growth and its path to artificial general intelligence, or AI that can automate most economically valuable tasks.
“We cannot do it if we don’t have the compute. We’re so compute constrained,” said CEO Sam Altman in a podcast interview published Thursday. “If we had double the compute, I think we’d double the revenue right now.” Altman didn’t elaborate.
Potential investors in OpenAI will likely be eager to see any margin improvement related to OpenAI’s paying users, as higher margins on paid plans could help subsidize free users. Even so, the 70% compute margin figure remains well below the equivalent margins of publicly traded software companies, which can serve additional users (including free ones) at very low cost.
Another problem involves Google, OpenAI’s biggest chatbot competitor. Google uses custom server chips, tensor processing units, that lower its costs, while OpenAI uses pricey Nvidia server chips. As a result, OpenAI leaders believe Google is running AI much more efficiently, The Information reported. That means Google could be under less pressure to monetize its nonpaying users.
Earlier this year, OpenAI prioritized reducing its server costs to run its AI products, also known as inference. In February, it declared an internal “code red” after the launch of a new model from China’s DeepSeek, said a person with knowledge of the effort. DeepSeek’s developer claimed it cost a lot less to train the model than proprietary models from companies like OpenAI.
OpenAI also improved its compute margins because the cost of renting computing power fell throughout the year, and because it tweaked its AI models to run more efficiently, according to another person with knowledge of its figures. And the company started a higher-priced subscription tier, which brought more revenue from a subset of customers.