After eight posts watching the fight against referees, let’s do the constructive thing and actually describe one. Not a fantasy ministry with ten thousand employees and a flag. The minimum viable whistle. It fits on a napkin, and the fact that it fits on a napkin is the tragedy of this entire story.
First item: capability testing before release, done by outsiders. Not the maker’s word, not a blog post about internal red teams. Someone whose paycheck doesn’t depend on the launch gets real access, probes the system for the things that matter, deception, autonomy, dangerous skills, and holds the power to say not yet. That last clause is everything. Testing without the power to delay isn’t oversight, it’s theater with clipboards. We certify planes this way. We trial drugs this way. Nobody calls it tyranny at the pharmacy.
Second item: liability that follows harm. This one needs no new agency at all, just an old idea pointed in the right direction: if your system causes damage, you pay, and the black box is your problem, not your excuse. Liability is the referee that never sleeps and hires no staff, because it deputizes the insurance industry, and insurers, unlike regulators, actually check. Nothing on Earth focuses an engineering culture like a price tag on failure. Right now the price of shipping a dangerous capability rounds to zero, and you get exactly the caution that price buys.
Third item: transparency where it counts, which is not the marketing kind. Incident reporting: near-misses logged and shared, the boring aviation habit that made flying the safest thing you do, because every close call teaches the whole fleet instead of embarrassing one crew. Disclosure of the very largest training runs, so someone outside the building at least knows the biggest experiments exist. And audit access for the testers from item one, because you cannot referee a game played entirely in the dark with the lights controlled by one team.
Fourth item: an off switch that’s real. Registration of the handful of largest runs, the technical ability to halt them, and a legal authority somewhere on the planet empowered to give that order. Not for your laptop, not for a startup’s chatbot. For the few systems that concentrate more optimization power than anything our species has built. If that sounds extreme, notice we demand exactly this from every nuclear plant, and the operators drill the shutdown until it bores them. Nobody calls the control room a dictatorship.
That’s it. That’s the napkin. Outside testing, liability, incident reports, an off switch for the biggest runs. Notice what’s not on it: no ban, no pause, no ministry of permissible math, none of the strawmen from the loud debate. Every single item is borrowed from a field that already runs it, at costs that are rounding errors next to the budgets involved. These companies spend more on launch events. Medicine under trials still cures things constantly, in case anyone tries the innovation bedtime story on you again.
Which brings us to the part I find genuinely hard to write. None of this is undiscovered. The napkin has been sitting in plain sight for years, sketched by researchers, floated in hearings, nodded at by the very people racing past it. The gap between what serious referees would do and what exists is not a knowledge gap. We know the list. It’s a power gap: the people who want the list have no leverage, and the people with leverage are, at this very moment, spending fortunes to keep the napkin a napkin. Every month it stays one, the integration deepens, the too-late argument strengthens, and the window narrows a little more. It was possible. It is, for now, still possible. That’s not comfort. That’s a deadline.
Tonight, reread the four items, slowly. Outside testing. Liability. Incident reports. An off switch for the biggest runs. Now ask yourself which one you’d honestly call radical. Then ask why a version of each already exists for planes, pills, and power plants, and what exactly makes minds the exception. Sit with how hard that last question is to answer. I couldn’t. That’s why this stretch of the series exists.