Here’s a fact about this industry that would sound deranged anywhere else. Many of the people building the most powerful AI systems will tell you, openly, sometimes with a number attached, that they believe there’s a real chance the technology ends in catastrophe. Not critics. Not protesters with signs. The builders. Ask over coffee and you’ll hear odds like one in ten. Then they finish the coffee and go back to work.

Try that in aviation. Picture an engineer announcing a ten percent chance our planes eventually kill everyone, then heading in to make them faster. There’d be hearings. In AI it’s a personality trait, practically a conversation starter.

I’ve spent enough time around this industry to promise you these aren’t monsters. Most of them are thoughtful, kind, and alarmingly good at board games. So the interesting question isn’t how could they. It’s what lets them. And the answer is a stack of very human mechanisms that you and I run every day, just with lower stakes.

First, it’s the most interesting problem alive. Never underestimate that pull. The physicists who built the first terrible weapons wrote later about the sweetness of the problem, how the physics itself dragged them forward. Minds that might exceed ours is the sweetest problem ever posed. Telling a brilliant person not to work on it is like telling water not to find the crack.

Second, the office is normal. Existential risk does not survive contact with standup meetings and the snack wall. Your Tuesday isn’t the fate of the species. It’s a bug ticket, a design review, a colleague’s birthday cake in the kitchen. Doom is abstract. The cake is right there.

Third, everyone’s piece is small. Nobody builds the scary thing. One person tunes infrastructure, another cleans data, a third makes the interface friendlier. Each piece is defensible, even boring. The scary thing is only the sum, and the sum is nobody’s job.

Fourth, someone down the hall is handling safety. There’s a whole team. Their existence works like absolution. You can worry less, because worrying has been staffed. Whether that team can actually stop anything is a question politely left unasked, and we’ll get to their situation later in this series. Bring tissues.

Fifth, and this is the one I find most human, proximity feels like control. Being near the dangerous thing feels safer than reading about it from far away. If I’m in the room, the thinking goes, I can steer. The trouble is that the room, on inspection, contains no steering wheel. It contains a gas pedal and a very good sound system.

Add the money and status, which do their quiet work in every industry, and you get people who can hold two things at once with complete sincerity. This might end everything, and I’m shipping on Thursday.

That’s what actually unsettles me. Not hypocrisy. Sincerity. If they were lying about the worry, we’d have an ordinary scandal, and we know how to handle those. Instead we have something stranger, proof that humans can believe in a cliff and accelerate toward it, provided the road is smooth, the company is excellent, and everyone else is accelerating too. No alarm goes off in the human brain for that. Evolution never needed one.

Tonight’s experiment. An offer letter arrives. Triple your current salary. The most fascinating work of your life, beside the smartest colleagues you’ve ever had. And one clause, written in invisible ink: you privately estimate a one in ten chance the field you’re joining ends in disaster for everyone. Do you sign? Take your time. Be honest. Then consider that thousands of people got exactly that letter, and almost all of them signed.