The AI Bosses Want to Slow AI Down

The AI bosses want to slow AI down.

Who benefits? Funnily enough, the AI bosses.

Dario Amodei (head of Anthropic) published an essay called We Must Pace the Frontier. It’s well worth a read. Sam Altman backed him. Elon Musk said "Dario is right." Demis Hassabis supports the direction too.

These people can barely agree to hold hands at a conference. Now they all want to slow down the technology they are racing to build….

The timing is very convenient.

What Dario is asking for

Dario says the danger has increased because AI is starting to help build the next generation of AI. OpenAI says its coding agents now contribute 3.14 agentic workdays for every human research day.

I went through the different levels of AI building AI here. We are some distance from a model vanishing into a server and upgrading itself overnight. The labs are already using AI to speed up AI research though. And that could lead to RSI (recursive self improvement).

Jacob Coxon’s very public resignation, followed by the media circus, has pointed to this being a sensitive issue right now. Dario’s essay was released in this environment and is worth reviewing.

Dario's proposal has three parts: i) Put independent evaluators inside the frontier labs with employee-level access. ii) Coordinate the rules across democratic countries. iii) Then try to reach a global agreement that includes China.

Anthropic has committed to the first part. The evaluators would get desks, badges and access to internal systems. They would also be allowed to publish their findings, although Anthropic could redact security, legal, customer and commercially sensitive information.

Anthropic are also calling on the other Western labs (ie. OpenAI and Google) to join suit and pace the speed of development. The Western labs have provisionally said yes.

But…China.

Dario spends the whole essay calling China “authoritarian” and making them out to be the big threat.

His language does not suggest he wants to work with them at all.

Which brings into question whether this whole essay and strategy is genuine - or just a ploy to come out as the defenders of the West, say that they’ll pace AI and save the day as long as China agrees to knowing full well that China won’t play ball.

This means there’s no real risk of having to slow down the pace of development. But at the same time Anthropic and OpenAI get to say “well, we tried!”

Smart plan.

And Trump already making his position clear: America should win the AI race.

That gives everybody a very convenient loop. We would slow down if China did. China probably won't. So we cannot.

Then slow down

David Sacks gave the obvious reply. If OpenAI and Anthropic truly believe their systems are too dangerous, they can slow down tomorrow. Nothing in current law forces them to keep racing.

So: do it? Slow down. You are the frontier. What are you waiting for?

Instead, Amodei’s proposal creates a new system that every lab would need to satisfy. And then says the whole world has to agree with that.

And this harms smaller labs too. Cohere CEO Aidan Gomez called it a cartel. His complaint is simple: employee-level access and expensive compliance are manageable for Anthropic, OpenAI and Google. Smaller labs and open-weight builders may find the same rules much harder to meet.

The safety concern can be genuine. The rules can also protect the companies already at the top. Both can be true. And in this case that seems to be what Anthropic are angling for - let us set the rules by which the rest of the industry abides.

I've put Dario's proposal, the replies and the full argument in the video.

To the task,

Kyle