A financial services company shipped a credit decisioning model that started rejecting credit-worthy applicants from a specific zip code. When the incident got escalated, the conversation went like this:

“How did this happen?”

“The data was biased.”

“Why was the data biased?”

“The historical training data reflected past lending patterns.”

“Who owns the data?”

“The data team.”

So the data team got blamed. The data scientist who built the model got blamed. The incident was closed as a “data quality issue.”

But that’s not what happened. What happened was a business decision: to use historical lending data to train a credit model, knowing that the data contained biases from past discrimination. And nobody made that decision consciously. Nobody even knew it was a decision.

This is the accountability inversion. We blame the people who execute (the data team) and ignore the people who decided what to build (everybody else). And we do this systematically, in almost every company I’ve worked with.

How the Inversion Works

Here’s the mechanism. You have a process: data sourcing → data cleaning → model training → validation → deployment.

Someone (product, business, leadership) decides what problem to solve. Someone in product/engineering decides how to approach it. The data team gets requirements. They execute against those requirements. A model comes out. It gets shipped.

Something goes wrong.

Everyone looks back at the process. The data team was the last team that touched the model before it broke. So the data team gets blamed.

But the failure wasn’t in execution. The failure was in the decision about what to build.

If the requirements were: “Build a model using historical lending data to predict credit-worthiness,” and the data team said, “This data contains systemic bias,” and nobody responded by changing the requirements, then the person who ignored that flag made a decision. They decided to accept the risk of bias to get a faster model. That’s a legitimate decision. But it’s not the data team’s decision. It’s the business’s decision.

Instead, what happens is the data team raises the concern, the conversation stops, and when the model breaks, people say: “The data was biased.” As if bias is a natural property of data rather than a decision about which data to use.

The accountability inversion happens because people at the decision point don’t see it as a decision. They see it as “running the process.” The data team sees it as a constraint. And when something goes wrong, the constraint gets blamed.

Why This Matters More Than You Think

This matters because companies optimize based on where they assign blame.

If you blame the data team for bias, your company will hire better data scientists and invest in better data tools and build better data governance. These are all good things, but they don’t prevent the problem you actually had.

The problem you had was: someone built a model that discriminates against a protected class. That’s not a data problem. That’s a decision problem. That’s a product decision, a business decision, and a risk decision.

If you blame the data team, you’re saying: “The process failed because the execution wasn’t good enough.” So you make the execution better.

If you actually solved the problem, you’d be saying: “The decision-making was wrong. We made a conscious choice to use biased data and didn’t have the governance to catch it before shipping.” So you change the decision-making, not the execution.

And this inverts the incentives. The data team, knowing they’ll be blamed if something goes wrong, becomes more conservative. They push back harder on any requirements that seem risky. The business becomes more frustrated with data teams, because they slow everything down. The relationship becomes adversarial. And you end up with governance theater again.

Meanwhile, the actual decision-makers feel less pressure to think carefully about what they’re building. The worst outcome gets caught by the data team, so their job is to avoid shipping anything the data team flags. That’s a low bar.

What Actually Went Wrong

Let me walk through what should have happened in the credit model case.

Product says: “We want to predict credit-worthiness.”

Engineering/Data says: “We can do that. Here are the approaches: – Option A: Use historical lending data (fast, cheap, but contains systemic biases from past discrimination) – Option B: Use transactional data and alternative credit indicators (slower, more expensive, but less biased) – Option C: Use only recent lending data and accept that we have limited historical performance (somewhere in between)”

Then someone (product, business, risk, compliance) decides which option. That’s the decision. That’s the point where someone is making a choice about which risks to accept.

If you choose Option A, the decision is: “We’re accepting the risk of discriminatory outcomes in exchange for speed and cost.”

That’s a legitimate choice. You might make it. But it’s not a hidden choice. It’s an explicit decision that someone needs to own.

What usually happens instead is that the decision gets bundled into “let’s build the model” and nobody explicitly decides which option to take. The data team gets some vague requirements, picks the fastest path, and builds the model. When it breaks, you blame the execution.

The Three People Who Made the Decision

Here’s who actually decided to accept the bias risk:

1. The product person who framed the problem. If they framed it as “predict credit-worthiness from historical data,” they’ve already chosen Option A. They made a decision about data sources without realizing it.

2. The engineering/data person who translated the requirement into a concrete approach. They could have pushed back and said “this data is biased; here are the implications.” If they didn’t push back, they made a decision to accept it. If they did push back and were ignored, then the person who ignored them made the decision.

3. The business/leadership person who accepted the timeline and cost of Option A. If you decide your risk tolerance is “whatever the data allows” rather than “we need to actively manage bias,” that’s a decision with real implications.

Nobody is evil here. Nobody is incompetent. But three different people made pieces of a decision without any of them explicitly owning the whole decision. So when the model breaks, everyone points at someone else.

The data team gets blamed because they’re the last person in the chain. But they didn’t decide to use biased data. They were given a requirement and executed it.

What Accountability Actually Looks Like

Accountability means: someone explicitly decided and someone explicitly owns the outcome.

In the credit model case, here’s how it should work:

Before shipping: – Product frames the problem. (Decision 1: what are we solving?) – Product/Engineering present options with tradeoffs. (Transparency: here’s what each option actually means) – Someone (let’s call it Risk or Product leadership) decides which option. (Decision 2: which risk do we accept?) – Data/Engineering executes to that decision. (Execution with clear constraints)

If something goes wrong: – If the model breaks because of a data quality issue (execution failure), the Data team owns it. – If the model breaks because of a decision we made (the bias was predictable from the approach), the person who made the decision owns it.

The accountability framework clarifies who decided what. It doesn’t blame people for circumstances they didn’t choose. But it also doesn’t let decision-makers hide behind “the data was biased.”

The Rationalization That Kills Accountability

I’ve watched this happen in almost every company:

Something goes wrong. Someone asks: “Whose fault is this?”

The convenient answer is always “the process failed” or “the data was bad” or “the execution wasn’t good enough.”

The hard answer is: “We made a decision about what to build that we should have thought through more carefully.”

So companies invest in better processes, better data tools, better execution frameworks. And the same decision-making failures happen again, just with better process theater around them.

What changes when you flip the accountability is that decision-makers have to think about what they’re actually deciding. It’s not: “Do we want to move fast?” It’s: “If we move fast using this approach, what are the actual implications? Who’s the decision owner if those implications are bad?”

That conversation is uncomfortable. It’s easier to blame the data team.

The Pattern to Watch For

If your company has a pattern where the data team is blamed when ML models break, or where technical teams are blamed for bad outcomes, or where execution is blamed for strategy failures, you have an accountability inversion.

The fix isn’t better data governance or better technical execution (though those might help). The fix is clarity about who decided what.

Stop asking: “What went wrong with the execution?”

Start asking: “What decision was made about what to build, and who made it?”

And when you get that clear, you stop blaming the data team. You either blame the person who made the decision badly, or you stop treating it as a blame situation and start treating it as a learning situation.

Actually, you probably want to stop treating things as blame situations anyway. But first you have to stop blaming the wrong people.