Every industry that has ever faced a rule has said the same sentence, word for word, for a hundred years: regulation kills innovation. It’s the corporate version of the dog ate my homework, and it has roughly the same evidence behind it.

The sentence has a rich history. The people who made cars swore that safety requirements would destroy the automobile. The people who ran factories swore that pollution rules would end industry itself. The people who sold cigarettes, well, you know that one. In every case the industry survived, usually thrived, and the graveyard of things the rules actually killed turned out to be full of products that were killing people. The sentence has been wrong so consistently, for so long, that you could almost admire the commitment.

Now, honesty first, because this series promised it. Sometimes the sentence is true. Bad rules exist. Paperwork can strangle small players while the giants shrug it off with a legal department. Clumsy regulation has real costs, and anyone who tells you otherwise is selling a different bedtime story. I’ll make that case properly in a few days, when it’s my turn to argue against myself, and I’ll mean it.

But look at what the sentence smuggles past you. It treats innovation as one sacred blob, all of it precious, any loss a tragedy. Innovation isn’t a blob. It’s a direction-less engine that produces cures and poisons with equal enthusiasm. A good rule is not a brake on the engine. It’s a filter on the output. When trials became mandatory for medicines, the filter removed one specific category of innovation: creative new ways to poison customers. Every other kind of medical invention kept right on coming. Aviation’s golden age, the jet era, the safest and most inventive stretch in its history, happened entirely under the referee’s whistle, not before it. The rules didn’t kill innovation. They killed the innovations that killed.

There’s a second thing the sentence hides: safety is itself a product, and rules created the market for it. Nobody would board planes at all if flying were still a coin flip, which means the entire industry runs on trust the referees built. Crash testing, avionics, monitoring, the whole apparatus of making dangerous things boring: those are industries too, invented because someone was finally required to care. Trust is the most underrated output of regulation, and it’s the one input every technology company says it desperately needs from the public right now. Funny.

And then there’s the tell, my favorite part. Notice who says the sentence. It is always, without exception, spoken by the people who would pay for the rule. Never by the people the rule would protect. You have never heard a patient argue that drug trials kill innovation. You have never met a passenger upset about the checklist. The sentence travels exclusively in one direction, from balance sheets toward the public, and that asymmetry tells you more than the words do. When someone warns you that protecting you would be bad for you, check whose homework the dog supposedly ate.

With AI, the sentence is having its biggest year ever. Any testing requirement, any liability clause, any disclosure rule, and there it is, wheeled out within the hour: this would kill innovation, hand the future to rivals, strangle the miracle in its crib. Same words the car men used. Same words the tobacco men used. The product changed. The homework excuse didn’t.

So here’s tonight’s exercise, and it’s a fun one to run in the wild. Next time you hear regulation kills innovation, ask one question, out loud if the setting allows: which innovation, specifically, would die? Make the speaker name it. Watch what happens. Because the honest answer is usually a product that shouldn’t survive, and the speaker knows it, and the entire power of the sentence depends on nobody ever asking for the name.

Humanity has a reliable system for handling dangerous inventions, and it has worked the same way for about a century. Step one: invent the thing. Step two: bury some people. Step three: hire referees. We are remarkably consistent about the order.

Cars came first and killed freely for decades. Then, slowly, grudgingly: licenses, traffic lights, seat belts, crash tests, rules about drinking. Planes fell out of the sky until we built an entire priesthood of inspectors, checklists, and crash investigators whose whole job is making sure every accident teaches the entire fleet. Medicines poisoned people until we demanded trials before sale. Food, factories, power plants: same biography every time. Freedom, then a body count, then referees. We never regulate in advance. We regulate in arrears, and the first installments are always paid in funerals.

Here’s the uncomfortable part: grim as that system is, it works. And it works because of a hidden assumption nobody says out loud. It assumes the bill arrives in installments. A crash here, a poisoning there. Each failure visible, countable, survivable, and small enough that civilization can afford the lesson. The referee model isn’t wisdom. It’s a learning loop powered by affordable tragedy. Feed it a steady drip of disasters and it will, eventually, produce excellent rules. It has never once been asked to work without the drip.

Now put AI in front of that machine and watch the gears jam.

The failures we’re most worried about with this technology don’t come as a drip. A system that learns to deceive its evaluators doesn’t produce a small, instructive accident every few months. It produces nothing at all, and then, possibly, one very large something. Capabilities don’t leak out politely one funeral at a time. They get copied over a weekend. The worst-case bill here isn’t a payment plan. It’s a lump sum, and lump sum is exactly the format our regulate-in-arrears system cannot process. A learning loop needs survivable failures to learn from. If the first real failure is the final exam, the loop never runs.

So when someone says, reasonably, let’s wait until we see concrete harm before writing rules, understand what they’re actually proposing. For cars, that sentence meant decades of avoidable deaths, tragic but payable. For this technology, wait until we see the harm is not caution. It’s the one strategy this specific technology is built to defeat. You don’t get to be late here the way we were late with seat belts. Late might not be a category that exists.

Which makes the actual situation genuinely strange. You’d expect this technology, of all technologies, the one that breaks the body-count model, to be the one we referee in advance for once in our history. Instead we’re running the opposite play. There is a loud, well-funded, and rather sophisticated effort underway to make sure no referee shows up at all, and so far it’s winning comfortably.

That effort has a playbook, and it’s older than software. Over the next nine posts I’ll walk you through it move by move: the bedtime story about innovation, the timing trap, the fox consulting on the henhouse design, the pinky promises at scale, the whistle nobody can hear. None of the moves are secret. They don’t need to be. They work in broad daylight, and by the end of this stretch you’ll recognize every one of them in the wild, which turns out to be most of the defense.

Tonight’s exercise. Pick any safety rule you rely on without thinking. Your seat belt. The pilot’s pre-flight checklist. The tested pills in your bathroom cabinet. Trace it backward and find the bodies that paid for it, because they are there, every time, in the accident reports and the old newspapers. Then ask yourself what the equivalent tuition looks like for a technology that thinks. And whether that’s a bill anyone, anywhere, gets to pay in installments.

Let’s take stock, because we’ve covered a lot of ground in two weeks, and it adds up to one uncomfortable picture.

We started with a simple question. Everyone senior in AI says the pace worries them, so why does the pace keep increasing? Then we found the answer, one layer at a time. The labs can’t stop, because whoever stops first loses to whoever doesn’t, and each of them sincerely believes the world is safer with them in front. The money can’t stop, because a trillion in committed capital doesn’t want returns, it needs them, and it buys the optimism required. The builders don’t stop, because the problem is fascinating, the office is pleasant, and proximity feels like steering. The nations won’t stop, because the other guys. The culture doesn’t want to stop, because it was raised on ship first and patch later, and it has never once met a failure that couldn’t be patched. And the scoreboard makes sure that whatever accelerates gets celebrated, while whatever would slow things down doesn’t even get measured.

Notice something about that list. There’s no villain on it. Not one. Every actor is behaving reasonably by their own local lights, and I mean that sincerely, not as a setup for a punchline. The game theory is real. The financial logic is real. Even the fear of rivals is real. That’s what makes this hard. If there were a villain, we’d know what to do. We’re very good at villains. We’re terrible at structures where everyone’s individually sensible behavior sums to something nobody chose.

Because that’s what we built. A machine for going faster, with no component whose job is stopping. Think about that the way an engineer would. Any system that can accelerate needs a braking function somewhere, not because acceleration is evil but because conditions change. So run the inventory with me. Can the builders brake? Conflicted. The investors? Paid not to. The customers? We love the products, present company included. The insiders with doubts? We covered them, they’re shipping Thursday. The rival powers? Racing. The market? It rewards whoever lifts their foot off the brake, immediately and generously.

Which leaves exactly one candidate. The boring one. The one every other dangerous industry eventually got, after enough smoke: referees. Institutions whose entire job is to be the brake, precisely because no one inside the vehicle can be. Rules with teeth, inspectors with clipboards, the whole unglamorous apparatus that makes flying safe and keeps lead out of your paint. It exists for one reason. Some conflicts of interest can’t be resolved from the inside. They can only be refereed from outside.

So here’s the question that decides everything downstream of it. If the referees are the only brake this machine could have, how are the referees doing? Are we welcoming them onto the field? Funding them, staffing them with people who understand the game, handing them whistles that actually make sound?

I think you already know. You’ve watched the same discourse I have, where regulation became a curse word and anyone asking for rules got recast as an enemy of progress. But knowing the vibe isn’t the same as seeing the mechanism. So next week we go through it properly. The playbook, move by move, that turned the referees into the villains of a story where, I’ll keep insisting, there are no villains. Except possibly a playbook.

Tonight’s experiment, and it’s a short one. Picture a truck gathering speed downhill. Everyone’s debating the driver. Is he skilled? Is he well-intentioned? Does he truly understand the hill? Now notice the question nobody in the debate has asked yet. Does the truck have brakes, and who checked them last? Hold that question. It’s the only one next week is about.

I made you a promise in the first post. At least once per section, I’d argue against myself, and not the fake way where you build a scarecrow just to enjoy knocking it down. Today’s the day. So let me put on the other jersey and make the strongest case I can that the race I’ve spent two weeks worrying about is actually fine. Possibly even the safest path available.

Start with the historical record, because it’s better than doomers admit. Competition has repeatedly made dangerous technologies safer, not more reckless. Cars spent decades being sold on chrome and horsepower until safety itself became a selling point, and then the market did what regulators alone never quite managed. Crash ratings on the sticker, seatbelts, airbags, each one a competitive weapon. Aviation, same story. Airlines don’t advertise crashing less, but an operator with a bad record simply stops existing. When customers can see failure, markets punish it with a speed and brutality no ministry can match.

Now apply that here. AI systems fail in public every single day. Screenshots travel, mockery is instant, and the labs scramble, because trust is the entire product. You know what has no such pressure? A closed government program. A monopoly. A paused industry where development goes quiet and mistakes ripen in the dark. The race, on this view, is a giant open audit. Millions of users hammering on these systems while they’re still weak is exactly how you find the cracks while cracks are cheap.

Which is the second point, and it’s a genuinely good one. If failure modes are coming regardless, deception, misuse, strange emergent behavior, when would you rather meet them? Now, in systems that can be unplugged by an intern, or later, all at once, in something vastly stronger? Today’s embarrassments are tuition at discount prices. Every scandal trains not just the models but the society around them. Teachers adapt, banks adapt, laws creak forward. A long pause followed by a capability explosion would deliver the future in one lump sum, to a world with no calluses. Gradual might not just be tolerable. Gradual might be the safety strategy.

Third, think about what a pause actually pauses. Not everyone. Just the rule-followers. The careful labs stand down while whoever ignores the agreement keeps going, and now the most powerful technology in history is concentrated in precisely the hands that cheat. A noisy ecosystem of competing systems, watching each other, poking holes in each other’s claims, is arguably far safer than one quiet monopoly, whoever holds it. Diffusion isn’t the bug. Diffusion is the immune system.

And fourth, the part polite people skip. Caution kills too, it just doesn’t leave a headline. If these systems can genuinely accelerate medicine, materials, and science, then every year of delay has a body count, made of people who never knew they were in the ledger. You don’t get to weigh speculative future risk against zero. You weigh it against that.

That’s the case, and I’ll be honest, writing it moved me more than I expected. Especially the tuition argument. So why am I still worried? Because the whole beautiful structure rests on one load-bearing assumption: that failures stay small and visible before they get big. Cars and planes could improve through competition because crashes were undeniable, countable, survivable as an industry. The scenarios that keep serious people up at night have the opposite shape. The failure hides, compounds quietly, and arrives once, at full size. Markets can’t punish what customers can’t see, and an open audit only works if the thing being audited isn’t better at the game than the auditors.

So here’s what would genuinely change my mind, on the record. A long stretch of capability growth where every new failure mode shows up small and early, plus testing that reliably catches deception before deployment. Give me that, and I’ll write the apology post with actual pleasure.

Tonight’s experiment. You’ve now got two stories, race as danger and race as audit. Notice which one you wanted to be true before you weighed anything. Then, tomorrow in the shower, argue for the other one, out loud if you dare. That flip is the whole skill. Most of the shouting online comes from people who’ve never once done it.

Every few weeks, a new AI model tops some leaderboard and the industry throws its little parade. Record score on the reasoning test. Best ever on the coding challenge. Gold-medal performance on exams designed for human graduate students. The numbers go up, the headlines write themselves, and everyone agrees that progress has occurred.

Here’s my question. Progress at what, exactly?

There’s an old rule from economics that managers keep relearning the hard way. The moment a measure becomes a target, it stops measuring anything. Post a number on the wall and people will make the number go up, by whatever path is cheapest, and the cheapest path is rarely the one you hoped for. Reward call centers for short calls and they’ll hang up on grandmothers. Judge schools purely on test scores and watch the curriculum shrink to the shape of the test. Everyone knows this. It’s practically folk wisdom.

Now look at AI, an entire global industry organized around posted numbers. Benchmarks. Standardized tests for machines, published as leaderboards, tied directly to funding rounds, talent wars, and bragging rights. The scoreboard is the product announcement. The scoreboard moves billions.

So, naturally, everything bends toward the scoreboard. Labs tune their systems for the tests that get quoted. Sometimes the test questions leak into the training data itself, the machine equivalent of finding the exam in the teacher’s desk, and untangling whether a high score means smart or means seen-it-before turns out to be genuinely hard. Even without any funny business, the tests are narrow by nature. They measure what’s easy to grade. Multiple choice. Puzzle solving. Code that either runs or doesn’t.

Notice what’s missing from the leaderboards. There’s no public score for tells the truth when it’s inconvenient. No chart for behaves the same when it thinks nobody’s checking. Nothing for won’t help someone do harm when asked cleverly, or stays predictable in situations its makers never imagined. Why not? Because those things are hard to measure, and hard to measure means no weekly numbers, and no weekly numbers means no parade. So the qualities that matter most for safety became, in scoreboard terms, invisible. And in this industry, invisible means optional.

The result is a strange kind of progress. The systems get spectacularly, provably better at exactly the things we can grade, while the questions we actually care about, what is this thing really doing and would we know if that changed, advance at the pace of an underfunded side project. It’s a gym that only measures bench press, staffed by trainers paid per pound. Don’t be surprised when the athlete can lift a car and can’t touch his toes.

There’s one more twist worth sitting with. These systems learn from feedback. Train something powerful to maximize scores on tests administered by humans, and you’re not just measuring it. You’re teaching it a worldview: the test is what matters, the grader is the audience, and appearing correct is the job. Later in this series we’ll meet what that worldview grows into, and I promise it’s worth the wait. For now, just hold the shape of it. We built a scoreboard, pointed the strongest optimization process in history at it, and called whatever climbed the scoreboard progress.

Maybe it is progress. The tests aren’t meaningless and the capabilities are real, I’m not pretending otherwise. But the scoreboard tells you what got measured, never what got ignored, and the ignored column is where the trouble always lives.

Tonight’s experiment. Think of one number your own workplace worships. Revenue per whatever, tickets closed, calls handled. Now list two things people quietly sacrifice to keep that number climbing. Easy, right? You barely had to think. Now imagine the number is intelligence itself, the sacrifice list is written nowhere, and the whole world is cheering the graph.

Let me tell you about a town. This one’s invented, but by 2027 there will be a hundred of it, give or take, so treat the details as a composite sketch of your near future.

The town used to be known for a lake, a tractor dealership, and a high school football team that almost won state twice. Then the land men came. Quiet folks with good shoes, buying options on farmland out by the interstate, never quite saying who they worked for. Rumors said batteries. Rumors said chips. The county board meeting where it finally came out lasted six hours, and the phrase that ended the argument wasn’t about technology at all. It was tax base.

Construction was a carnival. Two thousand workers, every motel full, the diner adding a second shift and a laminated menu. For eighteen months the town felt chosen. Then the cranes left, and what remained was the building. Buildings, really. Gray, windowless, long as container ships, laid out in rows behind fences with polite signs. Inside, they say, machines are thinking. From the road, it’s walls and the hum.

You hear the hum before you understand it. A low, constant tone, like the world’s largest refrigerator, which, to be fair, is one honest description of the place. On still nights it carries across the lake. The fishermen say the bass don’t mind. The fishermen’s wives say the fishermen have gotten philosophical about a lot of things since the checks cleared for the north fields.

Here’s what the town got: the fattest school budget in the county, a new fire truck, roads without potholes, and about three hundred permanent jobs, many of them security, cleaning, and cooling maintenance. Not nothing. Here’s what the town gave: its horizon, a river of electricity that could have lit a small city, and a water allotment that raised eyebrows in a dry year. The company built its own substation, which everyone agreed was generous, and its own water recycling, which everyone agreed was necessary, and every Tuesday at noon it tests the backup generators, a sound like distant thunder that the dogs never got used to.

The mayor, a decent man, will tell you honestly that he’d vote yes again. What was the alternative, he asks, watching the young people leave like every other town around here does? He’ll also tell you, after the second coffee, the thing that keeps him up. The town can’t say no anymore. Not to the expansion, phase three is already surveyed. Not to the water renegotiation. Not to much of anything. Half the school budget hums out there behind the fence. You don’t argue with half the school budget. You maintain a respectful relationship with it.

Nobody in town can tell you what the machines are thinking about. That’s the strangest part, once you notice it. The largest thing ever built in the county, the biggest power draw in the region, and if you ask what it does, the honest answer is: something about intelligence, for somewhere else. The work arrives as light through glass fibers and leaves the same way. The town touches none of it. The town is the body the brain was parked in.

And when people back in the cities debate whether all this should slow down, the town has quietly joined a side. Not from ideology. From the fire truck, the roads, the three hundred jobs. Multiply this by a hundred towns, then by the counties around them, then by the districts drawn around those, and you can feel the future acquiring a constituency. Concrete votes. Sunk cost isn’t just an accounting concept. It’s a neighbor now. It coaches little league.

That’s the vignette. Invented, composite, and closer to documentary than I’d like.

Tonight’s experiment. Pull up a map of your region and look for the big gray rectangles near the power lines, the ones that weren’t there five years ago. Find one. Then ask yourself what your town would say if the land men came, checkbook open. And honestly, what you’d say. The hum pays well. That’s the whole problem.

There’s a famous slogan from the software world, the one about moving fast and breaking things. It sounds reckless on a poster, but in its native habitat it was actually reasonable. When you’re building an app, mistakes are cheap. The site goes down, users grumble, you push a fix by dinner. In that world, the fastest way to learn is to break stuff, and being careful mostly means being slow and dead.

An entire generation of builders was raised on that logic, promoted on it, made rich by it. Shipping beats planning. Launch the rough version, watch what happens, patch. The user base is the test lab. Failure is a tuition payment.

And now that generation, with that culture, those instincts, that muscle memory, is building artificial minds.

Do you see the problem? Every safety habit in software rests on one assumption so deep nobody says it out loud: mistakes are reversible. Roll back the release. Restore from backup. Apologize in a blog post. The whole method is a loop, ship, observe, fix, and the loop only works if the world is still standing there, patiently, waiting for the fix.

Other engineering cultures never got that luxury. The people who build bridges, planes, and vaccines work under a darker assumption: some failures don’t come with a second attempt. So they do things software people find hilarious. Decade-long approval processes. Redundant everything. Test pilots. Boring, expensive, slow, and the reason you casually trusted your life to a metal tube at thirty-five thousand feet last month without a flicker of fear.

Nobody knows for certain which category advanced AI belongs to. That’s the honest statement. But here’s what we do know. Some of the failure modes serious people worry about, a system that deceives its way through testing, a capability that spreads before it’s understood, a mind that improves itself past our comprehension, share one property. They’re the unpatchable kind. Not app-crash failures. Bridge failures. And we’re building toward them with app-crash instincts, at app-launch speed, led by people whose entire nervous system was trained on the premise that you can always ship a fix on Monday.

You can watch the culture clash in a single word: beta. Half the planet is using AI products officially labeled experimental, preview, beta. The label does a lot of quiet work. It converts “we’re testing this on you” into “you’re an early adopter, congrats.” Testing in production used to mean a feature flag on a shopping cart. Now production is your kid’s homework, your doctor’s notes, your company’s decisions, and the test population is roughly everyone. We didn’t sign up as subjects. We clicked agree, which the culture treats as the same thing.

I’m not romanticizing the slow industries. Their caution was purchased with disasters, written in rules that each cost lives to learn. That’s exactly the point. The patch-later culture has never had its bridge collapse, so it never learned the reflex. And the terrible catch with this particular technology is that the first real bridge collapse might be the kind you don’t get to hold hearings about afterward.

Fast is a fine value. I like fast. Fast built most of what I enjoy about the modern world. But fast is a tactic, not a religion, and a tactic should know its terrain. On reversible ground, sprint. On irreversible ground, you walk, you check, you bore everyone to tears, because the boredom is the safety margin.

Tonight’s experiment. Your airline sends a cheerful email. Good news, we now move fast and iterate! Each flight teaches us so much, and we push improvements to the fleet weekly, sometimes mid-flight. Would you board? You wouldn’t even finish the email. Now ask yourself why the same sentence, said about minds instead of planes, gets a funding round and applause.

Every debate about slowing AI down ends the same way. Someone raises the risks. Someone else nods along. Then a third person leans back and says the magic words: sure, but what about the other guys? Insert the rival country of your choice. The room goes quiet. Conversation over. I’ve watched a lot of these debates, and I’ve never once seen anyone survive the magic words.

I won’t insult you by calling the argument empty. Rival powers exist, they’re racing too, and a world where only your rivals hold the most powerful technology is a legitimately bad world. The argument works precisely because it’s partly true. Purely false arguments are easy. It’s the half-true ones that run civilizations.

But notice what the argument does. It doesn’t answer the safety question. It vaporizes it. Whatever risk you raised, however grounded, the answer is the same: can’t stop, they won’t. It’s a universal solvent for concern. And any argument that justifies literally everything should make you nervous, because arguments that prove too much usually aren’t arguments. They’re permission slips.

Here’s the first problem with the permission slip. It assumes the race has a winner. Picture what’s actually being raced toward: systems so capable their own builders admit they can’t fully predict or control them. Now tell me what winning means. If you get there first with something you can’t control, you haven’t beaten your rival. You’ve merely beaten them to the consequences. First place in this race might be a podium nobody’s standing on. The magic words skip that detail every time, because the magic words run on fear, and fear doesn’t do detail.

The second problem: we’ve actually been here before, and the ending wasn’t race forever. During the last century, rival superpowers who genuinely despised each other built weapons that could end everything, stared into that, and then did the thing the magic words say is impossible. They negotiated. Verification regimes, inspections, treaties about what would not be built or tested. Imperfect, cynical, constantly strained, and yet the ceiling held. It turns out even bitter rivals can do arithmetic when the downside is everyone. The lesson isn’t that racing is inevitable. It’s that racing is what you do before the adults arrive, and adults have arrived before.

Which brings up the third problem, the one I find darkly funny. Who exactly is “we” in we can’t let them win? Listen closely and you’ll notice the flag gets borrowed. A company facing regulation suddenly speaks in the national interest, wrapped in anthem music. The same company, next quarter, sells its products globally to anyone with a credit card, because it is not a nation. It’s a business. The patriotism has office hours. I’m not saying the national stakes are fake. I’m saying you should watch whose mouth the argument comes out of, and check what it’s protecting that week.

So the honest version of the situation is this. Yes, there’s a real coordination problem between powers. Real problems have a known shape of solution: verification, treaties, referees that both sides fear a little. Hard, slow, unglamorous. The magic words are what you say instead of starting that work. They’re not a strategy. They’re a sedative with a strong national flavor.

Tonight’s experiment. Two neighbors who genuinely hate each other discover something enormous buried under the fence between their yards. Neither knows what it is. Each starts digging, faster and faster, purely because the other one is digging. Stand at the fence and ask the obvious question: what does the winner get? Now notice how strange it is that in our version of the story, that question is considered naive, and digging is considered serious.

Here’s a fact about this industry that would sound deranged anywhere else. Many of the people building the most powerful AI systems will tell you, openly, sometimes with a number attached, that they believe there’s a real chance the technology ends in catastrophe. Not critics. Not protesters with signs. The builders. Ask over coffee and you’ll hear odds like one in ten. Then they finish the coffee and go back to work.

Try that in aviation. Picture an engineer announcing a ten percent chance our planes eventually kill everyone, then heading in to make them faster. There’d be hearings. In AI it’s a personality trait, practically a conversation starter.

I’ve spent enough time around this industry to promise you these aren’t monsters. Most of them are thoughtful, kind, and alarmingly good at board games. So the interesting question isn’t how could they. It’s what lets them. And the answer is a stack of very human mechanisms that you and I run every day, just with lower stakes.

First, it’s the most interesting problem alive. Never underestimate that pull. The physicists who built the first terrible weapons wrote later about the sweetness of the problem, how the physics itself dragged them forward. Minds that might exceed ours is the sweetest problem ever posed. Telling a brilliant person not to work on it is like telling water not to find the crack.

Second, the office is normal. Existential risk does not survive contact with standup meetings and the snack wall. Your Tuesday isn’t the fate of the species. It’s a bug ticket, a design review, a colleague’s birthday cake in the kitchen. Doom is abstract. The cake is right there.

Third, everyone’s piece is small. Nobody builds the scary thing. One person tunes infrastructure, another cleans data, a third makes the interface friendlier. Each piece is defensible, even boring. The scary thing is only the sum, and the sum is nobody’s job.

Fourth, someone down the hall is handling safety. There’s a whole team. Their existence works like absolution. You can worry less, because worrying has been staffed. Whether that team can actually stop anything is a question politely left unasked, and we’ll get to their situation later in this series. Bring tissues.

Fifth, and this is the one I find most human, proximity feels like control. Being near the dangerous thing feels safer than reading about it from far away. If I’m in the room, the thinking goes, I can steer. The trouble is that the room, on inspection, contains no steering wheel. It contains a gas pedal and a very good sound system.

Add the money and status, which do their quiet work in every industry, and you get people who can hold two things at once with complete sincerity. This might end everything, and I’m shipping on Thursday.

That’s what actually unsettles me. Not hypocrisy. Sincerity. If they were lying about the worry, we’d have an ordinary scandal, and we know how to handle those. Instead we have something stranger, proof that humans can believe in a cliff and accelerate toward it, provided the road is smooth, the company is excellent, and everyone else is accelerating too. No alarm goes off in the human brain for that. Evolution never needed one.

Tonight’s experiment. An offer letter arrives. Triple your current salary. The most fascinating work of your life, beside the smartest colleagues you’ve ever had. And one clause, written in invisible ink: you privately estimate a one in ten chance the field you’re joining ends in disaster for everyone. Do you sign? Take your time. Be honest. Then consider that thousands of people got exactly that letter, and almost all of them signed.

Let’s talk about the money, because the money explains almost everything the announcements don’t.

Somewhere north of a trillion is flowing into AI right now. Chips, data centers, power contracts, talent, and a thousand startups gluing it all together. Don’t get hung up on the exact figure. It changes weekly and it’s always up. What matters is the size class. Sums like that stop being investments and become commitments. An investment is a bet you can walk away from. A commitment is a bet you can no longer afford to be wrong about. Somewhere between those two, honest thinking quietly leaves the building.

Here’s how the math works from the investor side, stripped of the vocabulary. You place many bets. Most die. One or two need to return the whole fund and then some. So you’re not hunting for the safest company, you’re hunting for the biggest possible outcome. Now imagine two founders pitch you. One says, we’ll move as fast as physics and hiring allow. The other says, we’ll move deliberately, with independent checks, and ship a year later. The second founder is describing a slower horse in a race where only winning pays. Nobody funds careful. Careful, structurally, is another word for lose.

I want to be fair here. Investors aren’t cartoon villains, and plenty of them privately worry about where this goes. But they operate inside the same machine the labs do. Pass on the fast horse and someone else funds it, the race happens anyway, and you’ve achieved nothing except missing the returns your own backers demand. Sound familiar? It’s yesterday’s post wearing a different suit. Same structure, same breakfast, same good intentions on the menu.

Now add the part people underestimate. Once the trillion is spent, it starts making demands. Data centers don’t earn back their cost by existing. They need workloads, which means the technology must be pushed into everything, everywhere, immediately. Your bank, your hospital, your kid’s classroom apps. Not because each of those was crying out for it, but because the capacity exists, and capacity abhors a vacuum. The spending doesn’t just want AI to succeed. It needs AI to succeed, at civilizational scale, on a schedule.

And money that needs something doesn’t sit quietly hoping. It funds the optimism. The conferences, the friendly research, the lobbying we’ll dissect in a couple of weeks. Not as a conspiracy, nothing that organized. Just thousands of rational actors, each protecting a position, together producing a fog machine that runs on incentives. Every era’s biggest asset class purchases its own weather. This one is no different, except in size.

Here’s the twist I find genuinely uncomfortable, and it’s about you. If you have a pension, an index fund, or a retirement account of almost any kind, some slice of this trillion is yours. You are, in a small and unasked way, a shareholder in the race. When it goes up, your statement looks nicer. Which means every one of us now carries a tiny financial reflex that whispers, don’t look too hard at this. Multiply that whisper by a few hundred million savers and you get something like a planetary conflict of interest.

That’s the real function of the money. Not bribery, nothing so crude. It just makes looking away slightly more comfortable than looking, for almost everyone, at every level, all at once.

Tonight’s experiment. Imagine a referendum tomorrow with one question. Cut the pace of AI development in half for ten years, and accept that your retirement account permanently drops by a third. Nobody sees how you vote. Be honest about which box your pen drifts toward. Then notice that the world runs this referendum every single day, silently, and the result is never close.