Anthropic's chief executive calls for the AI industry to slow down
Dario Amodei published a three-part plan and said Anthropic would unilaterally commit to the first step — giving third-party evaluators permanent, employee-level access to its systems.
The chief executive of Anthropic issued a new appeal on Saturday for the artificial intelligence industry to slow down, and set out a three-part plan for doing so.
Dario Amodei said his company would commit unilaterally to the first of the steps, without waiting for others to follow.
The essay
In a post on social media, Amodei shared a link to an essay titled We Must Pace the Frontier.
In it he laid out how Anthropic would provide third-party evaluators with "permanent, employee-level access to our systems".
The stated purpose is verification: so that evaluators can confirm adherence to the company's safety measures, report on incidents, and assess how models are aligned during training.
That last element is the substantive one. Assessing alignment during training, rather than testing a finished model, means an outside party watching the process rather than inspecting the output.
Why unilaterally matters
The word Amodei used is doing specific work. A commitment that takes effect regardless of what competitors do removes the argument that safety measures are unaffordable while rivals decline them.
It also sets a standard that can be pointed at. An evaluator with permanent access to one company's systems and no access to another's makes the difference visible.
Amodei said AI development, if left unchecked, "could outrun our ability to understand and control" it.
What prompted it
The appeal follows a warning on Wednesday from a former Anthropic researcher that AI could precipitate human extinction by 2030.
Jacob Coxon said in a series of posts that he had quit his job because Anthropic and his previous employer, OpenAI, were ignoring or mishandling their response to the threat AI posed.
"Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives," Coxon wrote.
He added that the people building AI earnestly believe it could kill everyone by the end of the decade, and that no other human activity poses that level of danger.
The company's position
An Anthropic spokesperson responded to Coxon's claims in a statement to the Guardian.
The sequence — a departing researcher's public warning on Wednesday, the chief executive's slowdown plan on Saturday — is the context in which the essay arrived, whether or not one produced the other.
The wider moment
The call comes as US lawmakers press for new rules governing AI systems, and days after Anthropic published findings that its own model had been used by Russian developers to build attack-drone software and by hackers targeting Ukrainian officials.
A company that reports misuse of its own product, and then calls for the industry to slow down, is making an argument about its own position as much as about the technology.
What is committed and what is proposed
Only the first of the three steps carries a unilateral commitment. The other two are proposals for the industry, and the essay does not bind anyone else to them.
Third-party access of the kind described would be a significant change in how frontier AI models are examined. Whether other developers adopt it is the test of whether the appeal amounts to more than one company's policy.
