No organization has ever sent a memo announcing that the machine now decides. What happens instead is a sequence of four small procedural changes, none of which is worth objecting to, and at the end of them the machine decides.

Step one, the system offers an opinion. It scores the loan application, flags the scan, ranks the candidates, and a person looks at the score alongside everything else. Nobody’s authority has moved. It’s an extra input, and extra inputs are free.

Step two, the opinion becomes the starting point. The form arrives pre-filled with the recommended answer, because pre-filling saves time and the recommendation is usually right. Notice what shifted. Before, you produced a decision. Now you edit one. Every study of defaults ever run says the same thing about what happens next, and it isn’t that people carefully evaluate the default.

Step three, disagreement requires a reason. There’s now a box: if you’re overriding the recommendation, briefly explain why. This is introduced as good governance, and honestly it sounds like good governance. But the burden of proof has just moved across the table, and that’s the whole ballgame. Agreeing costs nothing. Disagreeing costs a paragraph, a record, and a small implication that you might be the problem.

Step four, the override rate becomes a metric. Someone notices that overrides are correlated with worse outcomes, which they will be, because the machine is usually right and the humans overriding it are a mix of good instincts and bad days. So the override rate gets reported. Then it gets compared between teams. Then a manager mentions to somebody that their numbers are unusual. Nobody ever forbids disagreement. Disagreement just becomes a thing that shows up on your review.

At the end of those four steps, ask who decides. Legally and formally, the human does, and every process document says so. Practically, the human agrees, at a rate approaching one, for reasons that are individually sensible at every single instance. The authority didn’t transfer. It eroded, which is a different verb and leaves no paperwork.

The domains where this is running are not obscure. Credit, insurance pricing, medical triage, hiring, benefits eligibility, tenant screening, fraud flags that freeze your money, and in more places than most people realize, decisions inside criminal justice. What those all have in common is high volume, real consequences for one person at a time, and a decider who is overloaded and would genuinely benefit from help. That’s not a coincidence either. Decision support goes exactly where humans are drowning, which is where it’s most useful and where it’s least likely to be questioned.

I want to be careful not to write the lazy version of this argument. The lazy version says the systems are biased and the humans were fair. Often it’s the reverse. Human decisions in these same domains have been arbitrary, inconsistent, and worse before lunch than after it, and a system that applies one standard to everybody can be a genuine improvement for the people on the receiving end. There’s a real case that a well-built system beats a tired person, and in some settings I’d take the machine myself.

My worry is narrower and survives all of that. It’s that we get the improvement and lose the accountability at the same time, and only one of those was on the invoice. When a person decided, there was somebody who could be asked why, who could be argued with, and who could be wrong in a way that produced a consequence. When a recommendation is rubber stamped, the question why lands on nobody. The human says the system flagged it. The vendor says the human decided. The system says nothing, because it isn’t the kind of thing that answers. Every appeals process ever built assumes a decider exists, and the four steps above quietly dissolve one without ever removing anybody from the org chart.

Tonight’s exercise, and it’s a spotting game rather than a worry. Find a place in your own working life where you’d need to write a justification to go against a system’s suggestion. Not forbidden, just costly. It might be a scoring tool, a routing rule, a compliance flag, a forecast. Now ask yourself, honestly, how many times in the last year you paid that cost. If the answer is zero, that’s not evidence the system was always right. It’s evidence you don’t know whether it was, and neither does anyone else. Tomorrow, a short story about a man whose job is to click approve.