Where Are the Brakes on This Thing?

Let’s take stock, because we’ve covered a lot of ground in two weeks, and it adds up to one uncomfortable picture.

We started with a simple question. Everyone senior in AI says the pace worries them, so why does the pace keep increasing? Then we found the answer, one layer at a time. The labs can’t stop, because whoever stops first loses to whoever doesn’t, and each of them sincerely believes the world is safer with them in front. The money can’t stop, because a trillion in committed capital doesn’t want returns, it needs them, and it buys the optimism required. The builders don’t stop, because the problem is fascinating, the office is pleasant, and proximity feels like steering. The nations won’t stop, because the other guys. The culture doesn’t want to stop, because it was raised on ship first and patch later, and it has never once met a failure that couldn’t be patched. And the scoreboard makes sure that whatever accelerates gets celebrated, while whatever would slow things down doesn’t even get measured.

Notice something about that list. There’s no villain on it. Not one. Every actor is behaving reasonably by their own local lights, and I mean that sincerely, not as a setup for a punchline. The game theory is real. The financial logic is real. Even the fear of rivals is real. That’s what makes this hard. If there were a villain, we’d know what to do. We’re very good at villains. We’re terrible at structures where everyone’s individually sensible behavior sums to something nobody chose.

Because that’s what we built. A machine for going faster, with no component whose job is stopping. Think about that the way an engineer would. Any system that can accelerate needs a braking function somewhere, not because acceleration is evil but because conditions change. So run the inventory with me. Can the builders brake? Conflicted. The investors? Paid not to. The customers? We love the products, present company included. The insiders with doubts? We covered them, they’re shipping Thursday. The rival powers? Racing. The market? It rewards whoever lifts their foot off the brake, immediately and generously.

Which leaves exactly one candidate. The boring one. The one every other dangerous industry eventually got, after enough smoke: referees. Institutions whose entire job is to be the brake, precisely because no one inside the vehicle can be. Rules with teeth, inspectors with clipboards, the whole unglamorous apparatus that makes flying safe and keeps lead out of your paint. It exists for one reason. Some conflicts of interest can’t be resolved from the inside. They can only be refereed from outside.

So here’s the question that decides everything downstream of it. If the referees are the only brake this machine could have, how are the referees doing? Are we welcoming them onto the field? Funding them, staffing them with people who understand the game, handing them whistles that actually make sound?

I think you already know. You’ve watched the same discourse I have, where regulation became a curse word and anyone asking for rules got recast as an enemy of progress. But knowing the vibe isn’t the same as seeing the mechanism. So next week we go through it properly. The playbook, move by move, that turned the referees into the villains of a story where, I’ll keep insisting, there are no villains. Except possibly a playbook.

Tonight’s experiment, and it’s a short one. Picture a truck gathering speed downhill. Everyone’s debating the driver. Is he skilled? Is he well-intentioned? Does he truly understand the hill? Now notice the question nobody in the debate has asked yet. Does the truck have brakes, and who checked them last? Hold that question. It’s the only one next week is about.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.