Tokens may soon drive the AI economy
Jensen Huang of Nvidia has outlined a future based around the production, consumption and monetisation of output units
A new economic reality is starting to take hold in AI. It already underpins the industry’s giant data centres and it will one day become an iron rule for all companies that use machine-generated intelligence.
That, at least, is according to Jensen Huang, chief executive of Nvidia, who promoted the idea heavily at his company’s main annual tech event this week. His theory helps to make a case for Nvidia’s continued dominance in chips. But it also reveals how far the industry has to go to make a wider case for the technology.
Huang’s take on AI economics is based around the production, consumption and monetisation of tokens. These are the most basic units of output from large language models: it takes about 1,300 tokens to generate 1,000 words of text. The key metric, he argues, is the cost per token of output. And as the main input into AI-powered services, he adds, tokens translate directly into revenue.
It is not hard to see why the Nvidia boss wants a nervous Wall Street to focus on token economics. Forget the gargantuan capital spending or the fact that so many competitors are lining up to eat into Nvidia’s fat profit margins, he seems to be saying: as long as his company’s chips keep pumping out tokens at the lowest cost and as long as demand for tokens continues to far outstrip supply, then all is well with the AI boom.
As a theory of Nvidia’s continued pre-eminence, it sounds compelling. But if token economics is ever to rule the AI world in the way that Huang predicts, some important gaps need to be filled in.
One is the lack of a clear link between the production of tokens and the creation of value for customers. Just because the cost of tokens is falling doesn’t mean the services created with AI suddenly become valuable or that this will automatically generate revenue across the industry, as Huang suggests.
Complicating this picture is the fact that newer AI models consume far larger numbers of tokens. The “reasoning” models that emerged late in 2024, starting with OpenAI’s o1, perform far more work to arrive at an answer. These are now being supplemented by agents, which promise to automate white-collar work and bring an explosion in token use — and, by extension, hefty bills for companies that give workers unlimited use of AI.
Nvidia and the rest of the AI industry have barely scratched the surface when it comes to showing how this will translate into revenue for their customers. In software engineering, which has seen the first widespread use of AI agents, there have been efforts to measure how token use is linked to output and to use this to apportion tokens to workers. Eventually, tech companies dream of AI becoming a core part of employment, with the cost of all white-collar workers coming to be seen as a salary plus a certain number of tokens per month. For now, that is still only a pipe dream.
The second significant piece missing from Huang’s narrative about an emerging token economy is how the companies that produce tokens, the raw commodity on which all of this depends, will make profits. If these “AI factories” all use Nvidia’s latest chips, then it may be hard for any of them to gain a cost-per-token advantage or retain any pricing power.
The big price declines that have accompanied the plunging cost of producing tokens seem to bear this out. When OpenAI launched GPT-4 two years ago, for instance, it charged $33 for 1mn tokens. Today, it charges only 9 cents for 1mn tokens produced by its cheapest model. That may be great for customers, but it has fed worries about commoditisation.
Such worries are hardly new. It is the same argument that was heard in the early days of cloud computing, when Amazon Web Services charged for access to basic data storage and computing power. How could cloud companies ever make a decent profit if computing services were stripped back and sold as commodities like this? The answer was that these were only the first components of what became higher-value services — full-scale computing platforms on which customers could run their businesses. Whether OpenAI and Anthropic will be able to work a similar trick is unclear, but the opportunity before them is clear.
There may be other explanations for the healthy profit margins in cloud computing. The business is ruled by a small oligopoly. Cloud companies have also faced pressure from regulators to reduce switching costs that may help to pad their profits.
For now, there is no shortage of competition among frontier AI companies. How that shakes out in future will go a long way to shaping the industry’s profits.