Inside the AI Industry’s Behind-the-Scenes Push to Police Itself
The Takeaway
- Leading AI labs have been meeting to discuss an industry standards body.
- Top AI CEOs back industry-led self-regulation for AI development.
- Trump administration’s draft executive order for AI self-regulator stalled.
Anthropic, OpenAI and Google have been holding discussions about working together to create a standards body for the AI industry, people familiar with the matter said, even before Anthropic CEO Dario Amodei called on Saturday for AI companies to coordinate on testing and auditing the technology.
OpenAI CEO Sam Altman, for his part, told staff at a companywide town hall meeting earlier this week that he supported a testing and auditing organization for the AI industry but believed that major AI labs would have to create a standards body on their own without the support of the U.S. government, a person familiar with the comments said.
That kind of industry self-regulation was part of a broader proposal Amodei published on Saturday to slow AI development and establish independent oversight amid mounting risks posed by increasingly powerful AI. Amodei said companies could move ahead with voluntary safety standards while the government hammers out AI regulation, with the government potentially enabling or mediating industry discussions.
Amodei’s post on X quickly garnered endorsements from several other tech leaders including Altman, who wrote that slowing cutting-edge AI development had been a topic of discussion at his company in recent weeks. Demis Hassabis, who recently stepped down as CEO of Google DeepMind, said the proposal was overall a step in the right direction, though some details “need working through.” SpaceX CEO Elon Musk also endorsed Amodei’s plan.
The idea of a self-regulatory organization for AI has been part of the public conversation for several months. In July, Hassabis published an essay laying out a proposal for a self-regulatory body modeled after the Financial Industry Regulatory Authority, which oversees U.S. broker-dealers and securities professionals.
But the extent to which companies had already started working together on setting industry standards has not been previously reported. Since July, a working group of representatives below the CEO level from Google, Anthropic and OpenAI have been regularly meeting about the proposal for a standards body, according to people familiar with the matter.
An executive order draft circulated in the White House that would have created such an organization, but the effort stalled in the late summer after failing to garner enough support within the Trump administration, The Information previously reported. Former White House AI czar David Sacks has been a vocal critic of creating a FINRA-like body for AI, arguing against involvement from the government. Meta Platforms CEO Mark Zuckerberg also expressed his concerns about plans for a national AI regulator in an August phone call with President Donald Trump, Politico reported.
In the town hall this week, Altman said that while he had been optimistic the government would support a standards body, there was not yet enough industry consensus, and some companies opposed the idea, the person familiar with the comments said. He added that the best path forward would be to try to do something with the other labs. Altman told employees at the same town hall that OpenAI could also slow its pace of development, Bloomberg reported.
The working group meetings between Anthropic, OpenAI and Google about the industry-led standards body, which the people said are ongoing, signal appetite for action among the three labs beyond their executives’ high-profile public comments and proposals. The group met as recently as the past week, said one of the people.
It’s unclear where Meta stands on the issue. On Saturday, Alexandr Wang, the company’s chief AI officer, wrote on X that “alignment is fundamental to delivering personal superintelligence for everyone,” referring to the idea of AI systems’ behavior adhering to the intent of the directions provided to them, though he did not specify support or opposition to Amodei’s proposal.
Anthropic’s own employees drew fresh attention to concerns about the potential risks of increasingly powerful AI with posts that went viral on X in recent days. An Anthropic researcher said he was quitting the company because neither it nor OpenAI was “acting responsibly,” while a current Anthropic employee said the company genuinely believes AI could pose an existential threat to humanity.
On Saturday, Amodei proposed that each leading AI company should give outside evaluators such as Model Evaluation and Threat Research the same access to their technology that employees receive, and he said that Anthropic was committing to the step. He also called on the government to enable AI companies to coordinate on safety without running afoul of antitrust laws, saying the government should issue a “narrow waiver” for certain conversations.
Sacks said on X that AI companies should go ahead with the safety efforts and didn’t need an antitrust exemption to coordinate. But he questioned whether slowing development was “purely altruistic,” noting that product liability risks already provided a financial incentive to prevent AI from causing harm.
Some in the AI industry were more critical of Amodei’s plan. Cohere CEO Aidan Gomez raised concerns in a post on X about the largest AI companies coordinating on rules that could constrain smaller rivals, writing, “Great ideas here from the cartel.”