Every debate about slowing AI down ends the same way. Someone raises the risks. Someone else nods along. Then a third person leans back and says the magic words: sure, but what about the other guys? Insert the rival country of your choice. The room goes quiet. Conversation over. I’ve watched a lot of these debates, and I’ve never once seen anyone survive the magic words.
I won’t insult you by calling the argument empty. Rival powers exist, they’re racing too, and a world where only your rivals hold the most powerful technology is a legitimately bad world. The argument works precisely because it’s partly true. Purely false arguments are easy. It’s the half-true ones that run civilizations.
But notice what the argument does. It doesn’t answer the safety question. It vaporizes it. Whatever risk you raised, however grounded, the answer is the same: can’t stop, they won’t. It’s a universal solvent for concern. And any argument that justifies literally everything should make you nervous, because arguments that prove too much usually aren’t arguments. They’re permission slips.
Here’s the first problem with the permission slip. It assumes the race has a winner. Picture what’s actually being raced toward: systems so capable their own builders admit they can’t fully predict or control them. Now tell me what winning means. If you get there first with something you can’t control, you haven’t beaten your rival. You’ve merely beaten them to the consequences. First place in this race might be a podium nobody’s standing on. The magic words skip that detail every time, because the magic words run on fear, and fear doesn’t do detail.
The second problem: we’ve actually been here before, and the ending wasn’t race forever. During the last century, rival superpowers who genuinely despised each other built weapons that could end everything, stared into that, and then did the thing the magic words say is impossible. They negotiated. Verification regimes, inspections, treaties about what would not be built or tested. Imperfect, cynical, constantly strained, and yet the ceiling held. It turns out even bitter rivals can do arithmetic when the downside is everyone. The lesson isn’t that racing is inevitable. It’s that racing is what you do before the adults arrive, and adults have arrived before.
Which brings up the third problem, the one I find darkly funny. Who exactly is “we” in we can’t let them win? Listen closely and you’ll notice the flag gets borrowed. A company facing regulation suddenly speaks in the national interest, wrapped in anthem music. The same company, next quarter, sells its products globally to anyone with a credit card, because it is not a nation. It’s a business. The patriotism has office hours. I’m not saying the national stakes are fake. I’m saying you should watch whose mouth the argument comes out of, and check what it’s protecting that week.
So the honest version of the situation is this. Yes, there’s a real coordination problem between powers. Real problems have a known shape of solution: verification, treaties, referees that both sides fear a little. Hard, slow, unglamorous. The magic words are what you say instead of starting that work. They’re not a strategy. They’re a sedative with a strong national flavor.
Tonight’s experiment. Two neighbors who genuinely hate each other discover something enormous buried under the fence between their yards. Neither knows what it is. Each starts digging, faster and faster, purely because the other one is digging. Stand at the fence and ask the obvious question: what does the winner get? Now notice how strange it is that in our version of the story, that question is considered naive, and digging is considered serious.