Anthropic has long positioned itself as a safety-focused AI lab. Earlier this year, it refused to let the US military use its models for surveillance and fully autonomous weapons.
Anthropic has continued to release increasingly powerful models and, in February, walked back a key safety pledge, saying that it would no longer hold back potentially dangerous AI if rivals were close to matching its capabilities.
It was recently valued at $965 billion in a massive funding round and confidentially filed for a US initial public offering on Monday, putting it ahead of rival OpenAI in both valuation and the race to secure crucial funding.
Anthropic’s Thursday post cautioned that unilateral or poorly coordinated slowdowns could backfire if less cautious actors continue advancing, potentially reducing overall safety.
It said that a meaningful pause would require agreement among “multiple well-resourced labs” operating at the technological frontier, as well as rules on what conditions would trigger or lift such a pause and who would oversee it.
“A unilateral pause by one lab, by contrast, is achievable immediately, but accomplishes much less: it would change who the front-runner is, but it would not create the wider deliberative process that is currently missing,” the startup said.
Its research arm, Anthropic Institute, plans to study systems needed to support a slowdown and, in the coming months, will convene policymakers, researchers, civil society groups and rival AI firms to discuss managing risks such as recursive self-improvement.
OpenAI, xAI, Alphabet, Meta Platforms and France’s Mistral did not immediately respond to requests for comment on whether they would join the call.
Related
