What would multilateral ‘AI arms control’ look like?
Given the competition, it’s debatable whether a US-China safety deal is even possible
With the strictly limited release of OpenAI’s GPT 5.6 model last week, it isn’t only Anthropic that’s bearing the brunt of the Trump administration’s newfound zeal for regulating AI. Sam Altman, OpenAI chief executive, announced his company’s latest model while noting delicately that “this isn’t quite the process that we think is optimal”. Yet America’s AI regulation debate isn’t limited to the White House and leading AI labs. It is being shaped by competition with Beijing. Could China and the US agree to rules that would address the risks in advanced AI systems while in the middle of the race?
The frontier model contest changes nearly daily but Chinese companies continue to progress. The latest example is Z.ai’s newly released GLM 5.2 model, whose capabilities led David Sacks, co-chair of President Donald Trump’s Council of Advisors on Science and Technology, to declare: “We now have a Chinese open-weight model that is as good as the currently available models from OpenAI and Anthropic.”
In fact, model quality is growing harder to compare as the industry questions the efficacy of standard benchmarks such as mathematical problem completion. These benchmarks can be gamed if the model is trained on the specific problems in advance. What’s more, as OpenAI researcher Noam Brown recently noted, the right definition of capability is not only what problems a model can solve but how quickly and at what cost.
For complex, long-horizon tasks — the ones Anthropic and OpenAI are focused on — US companies may well have a longer lead than simple benchmarks suggest. But there’s no doubt Chinese models are improving. Before Trump, the Biden administration had hoped its AI chip restrictions on Beijing might give the US such a commanding lead in AI that Washington could dictate terms. This compute advantage has kept US companies ahead when measured by model capability or revenue, but it has not prevented Chinese competitors from releasing their own high-quality models. If Trump tightly regulates US companies he will kneecap American AI leaders without fully addressing global security concerns.
There is one alternative to unilateral regulation: a deal between China and the US to address AI safety. The Trump administration appears keen for talks with Beijing. The two sides reportedly discussed the topic before the May Xi-Trump summit, though Beijing has not agreed to formal negotiations. So it is worth asking what a multilateral “AI arms control” regime could look like.
To start, we need to take cyber security — the reason that the White House is restricting foreign access to Anthropic’s Fable model — completely off the table. It’s nearly impossible to imagine the two countries agreeing not to use AI for cyber operations because it’s already happening so widely. Both Chinese hackers and the US National Security Agency use leading Anthropic models for this purpose, according to reports. This isn’t surprising: any model good at computer programming will be good at identifying cyber vulnerabilities.
A second regulatory priority is limiting the ability of AI to enable production of dangerous pathogens. It’s not impossible to imagine that China and the US could develop shared principles for assessing a model’s biosecurity risk. However, China’s record of biosecurity transparency is not reassuring. Nor are the cold war arms control parallels. When the US and Soviet Union signed the Biological Weapons Convention in 1972, the US had already begun dismantling its bioweapons programme. But the Soviets, convinced the US was lying, expanded work on anthrax and smallpox.
Could AI controls extend into the sphere of weaponry?
China’s pre-existing support for talks on banning autonomous weapons may sound reasonable. But will any major power renounce use of autonomous missiles or automatic missile-defence systems?
An even bigger issue is assessing compliance.
If a country promised not to use AI for a specific purpose, how could this be proved? It was hard enough to count missile silos during the cold war, even though these were visible from space. Neither China nor the US will give the other access to sensitive source code.
“Trust but verify” was Ronald Reagan’s wise approach to US-Soviet talks in the 1980s. So long as we distrust and cannot verify, we should not expect a significant deal. Both Washington and Beijing are likely to conclude that the priority is to keep racing instead.