The easiest way to spot a governance framework that won’t work is to ask a simple question: “What happens when your training data is wrong?” Watch how long it takes for someone to answer.

Most organizations have governance structures for AI models that exist entirely downstream. They talk about model cards, testing protocols, access controls, deployment gates. These are real things. But they’re built on an assumption that the data feeding the system is adequate. It rarely is.

I’ve watched this play out in three different ways. First, there’s the organization that has excellent data quality standards—but only for their transaction systems. When AI teams pull data for training, they treat it like a novelty. There’s no service level agreement. No validation pipeline. No understanding of what “good” looks like. The AI governance meeting happens monthly. The data quality review never happens at all.

Second, there’s the organization that built data governance years ago around regulatory compliance. Their data lineage tools work great for financial reporting. But AI training pulls from sources those tools never contemplated—third-party datasets, unstructured feedback, behavioral logs. The data steward doesn’t know what’s in production. The AI team doesn’t know where it came from.

Third, and this one is common: the organization that has separate governance structures entirely. Data governance is a technical problem, owned by engineering. AI governance is a risk problem, owned by compliance. They email each other occasionally. The data person doesn’t understand why the model person needs certain attributes. The model person doesn’t understand why the data person won’t merge this dataset from another system. They optimize locally for their respective problems.

Here’s what actually happens in each scenario: Your AI system goes into production. Three months in, a downstream business process starts failing in ways that don’t quite make sense. You dig. You find that the training data included records from a system migration that was supposed to be flagged as an outlier period. Or the historical data contains behavioral patterns that don’t apply anymore. Or someone changed how a critical field was calculated, and that change wasn’t documented, so downstream models are using stale definitions. Or you discover that 15% of your training set was duplicated because someone ran a batch job twice.

By the time you realize this, the model is in production. You’ve built decisions on top of it. Your governance framework has no way to retroactively assess what changed. It has no way to answer: “Should we have caught this?” Because it has no real view of what the data actually was.

Proper data governance for AI means several concrete things. First, your data contracts have to extend all the way upstream to the source. You need to know not just what fields exist, but what they mean, how they’re calculated, when they change. This is harder than it sounds. A “date” field sounds simple until you discover it’s been calculated three different ways in three different legacy systems that were supposed to be consolidated in 2019 but actually weren’t.

Second, you need a testing mindset around data that matches your testing mindset around code. Before a dataset feeds a production model, it needs to pass checks. Are there unexpected nulls? Has the distribution changed? Are there new values that weren’t in training? This should be automated, not a checklist someone marks off once.

Third, you need traceability. Not just “we trained this model on this dataset,” but “this field came from this system, it was transformed by this pipeline, and these validations happened at each step.” When something breaks, you need to know the full lineage fast.

Fourth, you need an owner. Not a committee. Not a shared responsibility. Someone who wakes up in the morning thinking about whether the data feeding your AI systems is trustworthy. That person needs to have standing in governance conversations. When they say a data source isn’t ready, that’s a model-deployment blocker.

The hard part is that data governance doesn’t feel urgent until it is urgent. A model audit finds a problem, and suddenly you’re in crisis mode trying to figure out where your data actually came from. A regulator asks about training set composition, and you realize you don’t have a complete picture. A customer reports they were treated unfairly, and you need to reconstruct what inputs your model saw.

By then, your governance theater is exposed as purely ornamental.

The organizations doing this well don’t separate AI governance from data governance. They treat them as the same problem. The criteria for putting data into production are the same criteria for putting models into production. Same rigor. Same documentation. Same review. Same ownership. When you ask “How do we know this data is good enough?” you get the same answer you’d get if you asked “How do we know this model is good enough?”

That alignment is harder to build than any framework. But it’s the difference between governance that actually catches problems and governance that exists to check a box.

I visited two companies in the same week. The first had a centralized AI team. All AI decisions went through them. Nothing shipped without approval. The second had distributed AI. Every business unit built its own AI. The centralized company was slow. The distributed company was chaotic.

The problem isn’t centralization or decentralization. It’s that they’re both optimizing for the wrong thing.

The centralized company thought they were optimizing for control. What they got was bottleneck.

The distributed company thought they were optimizing for speed. What they got was risk.

Neither company understood the actual constraint: you need both speed and consistency. And the structure that gives you both of those isn’t about where you put the team. It’s about where you put the decision.

Why Centralization Doesn’t Work

A centralized AI team sounds good in theory. You have one group of experts. They set standards. They approve decisions. Nothing gets shipped that violates the standards.

What actually happens:

The centralized team becomes the bottleneck. Every business unit has to wait for the AI team to say yes. The business unit thinks: “We have a problem we could solve with AI. But it’s going to take six months to get approval.” So they either don’t solve the problem, or they find a workaround that bypasses the central team.

The central team thinks: “We’re protecting the company from bad AI decisions.” What they’re actually doing is training the organization to either wait or work around them.

Also, the centralized team can’t understand every business context. They make rules that work for some problems but not others. They say “we only use approved models” but different business units have radically different requirements. They say “we monitor for fairness” but fairness metrics are different for hiring vs. lending vs. content recommendation.

The company doesn’t end up with strong governance. It ends up with slow governance that’s frequently bypassed.

Why Decentralization Doesn’t Work

A decentralized structure sounds good on paper. Every business unit moves fast. They understand their own problems. They can experiment and iterate.

What actually happens:

Every business unit builds their own version of the same solution. The hiring team builds a resume screener. The recruiting team builds another one. The L&D team builds a third. None of them talk to each other. They’re all making different choices about data, models, monitoring, governance.

One team uses a model that’s biased against a protected class and doesn’t notice because they never thought to check. Another team’s system breaks down in production and they lose customer data because they didn’t think about operations. A third team’s decisions are unexplainable to regulators because they built a black box without documentation.

The company doesn’t end up with speed and flexibility. It ends up with fragmentation and invisible risk.

And the real problem: when something goes wrong in one business unit, the company’s regulatory exposure is enterprise-wide. The hiring team’s bias problem is a company problem. The recruiting team’s data loss is a company problem. The L&D team’s black box is a company problem.

So decentralization feels fast until it’s not. Then it’s expensive.

What Actually Matters Is the Decision Point

The real question isn’t “where does the team sit?” It’s “where does the authority sit?”

And here’s the shift that changes everything: authority should be at the decision point, not at the team location.

Let me explain what I mean.

In a centralized company, authority for all AI decisions is with the central team. A business unit comes and says: “We want to build a hiring screener.” The central team decides. That’s the problem. The central team doesn’t know enough about hiring to make that decision well.

In a decentralized company, authority is with the business unit. They decide. But the company has no way to ensure consistency or manage enterprise risk. A business unit makes a decision that creates regulatory exposure for the whole company, and the company can’t prevent it.

What actually works is splitting the authority:

The business unit decides: “We have a problem and we think AI can solve it.” That’s their decision. They know their problem better than anyone. They should have authority here.

A shared standards team flags: “If you use AI this way, here’s what you need to do around data quality, fairness, explainability, operational risk.” These aren’t approvals. They’re constraints.

The business unit decides: “We’ll meet these constraints with approach X.” That’s their decision. They know how to do it.

A shared audit function checks: “Are you actually meeting the constraints you committed to?” They’re not blocking. They’re verifying.

This distribution of authority gives you what you actually want:

• Speed: business units can move fast because they’re not waiting for approval
• Consistency: everyone is building to the same enterprise constraints
• Risk management: you’re checking for problems without blocking problems
• Accountability: it’s clear who decided what, and it’s clear why it was decided

How Standards Without Approval Actually Works

The shift from “approval authority” to “standard-setting authority” is subtle but important.

Approval means: “We get to decide if you can do this.” You’re slowing down the organization.

Standards means: “Here are the constraints. Meet them, and you’re free to move.” You’re enabling the organization.

The centralized company is doing approval. The distributed company is ignoring standards. Neither is working.

What works is: clear standards that business units must meet, clear authority for business units to decide how to meet them, and clear verification that they’re actually meeting them.

Here’s what this looks like in practice:

A business unit wants to build a pricing model. The standards say: “If you use AI for pricing decisions, you need to: – Document the data sources and any known biases – Verify that recommendations don’t systematically harm specific customer segments – Have a manual override process for edge cases – Monitor recommendation outcomes against actual pricing decisions – Review outcomes quarterly with your business leader and the standards team”

The business unit says: “We’ll do all that.” Great. They don’t need approval. They can start building.

Six months later, they’re in production. The standards team reviews: “Are they actually documenting data? Are they actually monitoring outcomes? Are they actually doing quarterly reviews?” Yes? Great. No? Then you have a conversation about what’s getting in the way.

The business unit didn’t get held up by approval. They also didn’t go rogue. They built to standards.

The Scaling Problem

The reason this matters is that your company is going to deploy AI across every business unit. Not maybe. It’s happening.

If you centralize, you’ll be the slowest company in your industry because you’re approving every experiment, every iteration, every new use case. Your competitors will be 10x ahead of you.

If you decentralize without standards, you’ll have governance risk in every unit, security risk in most of them, and regulatory exposure enterprise-wide.

The middle path is: clear standards, distributed authority, shared verification.

This requires:

1. Someone owns the standards. Not approves. Owns. Writes them. Updates them based on experience. Makes sure they’re actually good standards, not theater standards.

2. Business units have clear authority. They decide to use AI. They decide how to meet the standards. They move fast. They’re not waiting for approval.

3. Verification is real. Someone’s actually checking if the standards are being met. Not by reviewing documents. By checking outcomes, talking to teams, monitoring in production.

4. Escalation is clear. If a business unit can’t meet a standard, there’s a clear escalation. Not “you’re blocked.” But “here’s the constraint, let’s figure out how to address it.”

The Mistake That Kills This Approach

Most companies can’t do this because they try to start with standards that are too prescriptive.

They say: “All AI systems must use approved models.”

That’s not a standard. That’s an approval requirement hidden in standard language. And business units will either ignore it or spend six months trying to get a model approved.

Real standards look like:

• “All AI systems must have a documented rationale for the data sources.”
• “All AI systems used for consequential decisions must have outcome monitoring.”
• “All AI systems must have a person accountable for outcomes.”
• “All AI systems must have an escalation process if they start making decisions outside their training distribution.”

These are standards that you can verify. They’re not gatekeeping. They’re clarifying what good looks like.

The Organizational Structure That Enables This

You don’t necessarily need to reorganize. But you need clarity about authority:

Business unit leaders decide what problems to solve with AI. They have authority. They move fast.

A shared standards function (could be central AI team, could be part of Risk, could be part of Architecture) owns standards, not approvals. They set expectations. They verify. They escalate when needed.

An audit or verification function checks that standards are actually being met. This could be internal audit, compliance, or a dedicated team.

Escalation authority (could be CTO, CRO, business leader, doesn’t matter who) is clear. When a standard conflicts with a business need, there’s a clear path to resolve it.

This looks different from a pure centralization or pure decentralization. It’s designed for the actual constraint: you need speed and consistency simultaneously.

What You Should Do Monday Morning

If your company has a centralization-decentralization argument going, the question to ask is: “What are we actually optimizing for?”

If the answer is “we want to move fast,” decentralization feels right until you hit risk problems.

If the answer is “we want to manage risk,” centralization feels right until you slow down the business.

If the answer is “we want to move fast AND manage risk,” then the answer isn’t about where to put the team. It’s about where to put the authority.

Ask your leadership: “Where is authority for AI decisions in our company? Is it business units? Is it the central team? Is it clear?”

If you can’t answer in a sentence, you have a structural problem.

The companies that are scaling AI aren’t solving this with better committees or more governance layers. They’re solving it by being clear about where decisions are made and why.

Centralize authority where you need consistency. Decentralize authority where you need speed. But be explicit about where each one lives.

A regulator walked into a bank’s AI governance office and asked a simple question: “Show me your decision log for the AI systems you’re using in lending.”

The bank produced a spreadsheet: model name, version, accuracy metrics, deployment date.

The regulator asked: “Where’s the decision log? When did you decide this model’s output? When did it change? Why did you decide to use this model for this decision? Who approved it? What was the alternative?”

Blank looks.

What the bank had was a model registry. What the regulator was looking for was a decision record.

And the reason this matters is that the regulator’s job isn’t to check if your model is accurate. It’s to check if you made a reasonable decision about how to use AI in your business, and you can defend that decision.

Most enterprises aren’t prepared for that conversation. Not because they’re incompetent. But because they’ve been building technical infrastructure (governance committees, model monitoring, fairness frameworks) when they should have been building decision infrastructure.

Why Regulators Care About Decision, Not Accuracy

Here’s the shift that’s happening in regulatory thinking, and most enterprises haven’t noticed it yet:

Regulators care about what you decided to do with AI, not whether you did it well.

If you can show: “We decided to use AI in this part of our business because X. We evaluated the alternatives. We identified the risks. We have governance to manage those risks. We monitor outcomes. We’ve decided to accept these specific risks. Here’s how we track if we’re right.”

Then the regulator might actually approve. They might ask hard questions, but you have a story.

If you can’t show that—if instead you show: “We built a really good model. Look at the accuracy. Look at the fairness metrics.”—you’re not answering the question the regulator is asking.

The regulator already assumes your model is good. What they want to know is: Why did you use AI here? What problems could this cause? Have you thought about it? Who’s accountable if something goes wrong?

Most enterprises have built answer to “Is the model good?” They haven’t built an answer to “Did you decide to use AI responsibly?”

These are different questions, and they require different infrastructure.

What the Regulator is Actually Checking

When a regulator looks at your AI system, they’re asking:

Decision authority: Who decided to use AI here? Was that person qualified to make this decision? Did they understand the risks?

Alternatives considered: Did you evaluate other ways to solve this problem? Why did you choose AI? Why not just use rules? Why not keep the human in the loop?

Risk assessment: Did you identify what could go wrong? Have you thought about fairness, accuracy, security, operational risk? Or did you just think “this model is accurate”?

Monitoring and accountability: How will you know if something goes wrong? Who’s accountable? Do you have a way to fix it if you need to?

Documented reasoning: Can you show me the decision? Not the model performance. The decision about whether to use the model.

Most enterprises would fail this audit today. Not because your models aren’t good. But because you can’t show the decision.

You have model cards. You don’t have decision records.

What You Actually Need to Build

Here’s what “decision infrastructure” looks like:

Decision record template. Before you deploy an AI system, someone fills this out: What decision are we using AI to make? What are the alternatives? Why are we choosing AI? What could go wrong? What’s our monitoring plan? Who’s accountable?

This is boring and unsexy. Nobody gets excited about decision records. But when a regulator asks “how did you decide to use AI here?” you can point to a document that answers the question.

Audit trail. When you change a model, when you change monitoring, when you escalate a risk, you should have a record. Not “we deployed version 2 on this date.” But “we decided to change the monitoring thresholds because X, here’s who approved it, here’s the reasoning.”

This isn’t new. Banks have done this for lending decisions for decades. The infrastructure exists. Most enterprises just haven’t applied it to AI systems.

Escalation record. When something goes wrong with the AI system (accuracy drops, outcomes drift, fairness metric breaks), do you have a record of what happened and what you decided to do? That’s what regulators care about. Not that nothing went wrong. But that you caught it and did something about it.

Most enterprises have an incident response plan. They don’t have a “what did we decide when we found a problem with the AI system” plan.

Approval authority. When you want to deploy an AI system or make a material change to one, who approves it? Not a committee. A person. With clear authority. Who will own the decision if something goes wrong.

The Conversation Regulators Are Having

Regulators are starting to enforce this. It’s not consistent yet (different regulators care about different things), but the pattern is becoming clear:

“You have to be able to explain your AI decisions.”

This is not “you have to have a fair model.” This is “you have to be able to explain why you decided to use a model, what you evaluated, and what you’re doing to make sure it works.”

Some regulators care more about fairness. Some care more about security. Some care about operational risk. But all of them care about the decision.

And they’re discovering what I’ve seen in every enterprise: most companies can’t explain the decision. They can explain the model. The model is great. But the decision to use the model in the business? That decision was never explicitly made.

What This Means for Your Enterprise Now

If you’re waiting for regulatory guidance to clarify what they want, you’re late. Regulators are starting to conduct reviews now, and they’re finding the infrastructure gap.

What you should do:

1. Start with one model. Pick a model that’s already deployed. Walk through the decision. Who decided to use it? Did they document their reasoning? Do you have an alternatives analysis?

2. Build the decision record backward. You don’t have to have perfect decisions. But you have to be able to explain them. When a regulator asks “why did you use AI here?” you should be able to answer in a sentence or two.

3. Set up escalation tracking. When something changes with the model—accuracy drops, fairness metrics break, operational impact—do you have a record of what you discovered and what you decided to do?

4. Identify your approval authority. This is a person, not a committee. They can consult the committee. But they own the decision.

5. Plan for the hard conversations. When a regulator asks “can you defend this decision?” you might realize you can’t. That’s okay. You’re learning. But it’s better to learn this before a formal review.

The Difference Between Compliance Theater and Actual Compliance

Here’s the trap: you can build all the governance theater in the world—committees, frameworks, monitoring—and still not be able to answer a regulator’s actual questions.

Real compliance means: when a regulator asks “did you think about this?” you can say “yes, here’s what we thought, here’s what we decided, here’s how we’re managing it.”

Theater means: when a regulator asks “did you think about this?” you produce a document that says “we have governance” but doesn’t show that you actually thought about anything.

The companies that are handling this well are building decision infrastructure, not governance infrastructure. They’re keeping records of decisions. They’re being explicit about who decided what. They’re documenting their reasoning.

It’s not impressive. It’s not technically sophisticated. It’s boring process work.

But when a regulator asks for it, you have it. And that’s the difference between a review that goes smoothly and one that doesn’t.

The Regulatory Reckoning

There’s a regulatory reckoning coming. Not because regulators are anti-AI. But because regulators are starting to apply the same bar to AI systems that they apply to everything else: Can you defend the decision?

Most enterprises would fail that bar right now.

The time to fix this is now, not when you’re in a regulatory review. Build the decision infrastructure. Document your thinking. Make explicit decisions with clear authority.

Because when a regulator asks “show me your decision log,” you’re going to want to have one.

A bank’s Chief Risk Officer pulled out a model risk framework. It was comprehensive: data quality checks, concept drift monitoring, backtesting protocols, confidence intervals, stress testing. It looked good. Very academic.

Then I asked: “What happens when a model decides to drop a product line that’s currently profitable but the model thinks is risky?”

Silence.

“What if the model’s risk assessment is more conservative than the business risk tolerance, and nobody knows that’s the conflict?”

More silence.

“Who resolves that conflict?”

This is the gap that almost every enterprise is walking into: they’re using risk frameworks designed for research or trading (where “maximize accuracy” is the goal) and applying them to production systems in regulated industries (where “make the right business decision” is the goal, and accuracy is just one input).

These are not the same problem.

Why Academic Frameworks Don’t Translate

Let me be clear about what academic model risk frameworks do well:

They identify technical failure modes. They catch when training data has changed. They detect when model assumptions are violated. They flag when a model’s confidence is misplaced. These are real, important things.

What they don’t do is address business risk. They can’t, because business risk is not the same as model risk.

In a research setting, a model is either right or wrong. In an enterprise setting, a model is making a decision that has business consequences. The risk isn’t “the model is inaccurate.” The risk is “the model is making a decision that conflicts with our business goals or regulatory constraints, and we didn’t catch it.”

Here’s the concrete difference:

A research model that fails: “The accuracy dropped from 94% to 91%. Something is wrong with the model.”

An enterprise model that fails: “The model is rejecting 15% more credit applications than it used to. Is that because credit quality changed, or because we changed the training data, or because the model is malfunctioning, or because the model’s risk tolerance diverged from the business risk tolerance?”

The research framework will catch some of this (data changes, concept drift). It won’t catch the one that matters most: we didn’t check if the model’s decision-making is aligned with what the business actually wants.

The Misalignment Nobody Talks About

Here’s what I see in most enterprise risk frameworks:

“Monitor model performance. Flag degradation. Retrain. Validate. Deploy.”

What’s missing: “Validate that the model’s decisions are aligned with business goals.”

And the reason it’s missing is that people don’t think of it as a model problem. They think of it as solved by governance. “We have a governance committee that approves what the model does.”

But the committee approves the model. They don’t continuously validate that the model is doing what it approved.

So you get scenarios like this:

A model for employee promotion recommendations gets deployed. The governance committee approved it. The accuracy is good. The fairness testing passed. It’s in production.

Six months later, someone notices the model is recommending promotions for technical specialists into management roles at much higher rates than the business actually wants. The model learned that technical expertise is predictive of success, which is true statistically. But the business doesn’t want to promote all the good engineers into management. That’s not what the business risk tolerance is.

Is this a model risk problem? In the academic sense, no. The model is doing exactly what it was trained to do. In the business sense, yes. The model’s decisions conflict with what the business actually wants.

The academic framework checks: “Is the model accurate?” Yes. “Is the model fair?” Yes. “Is the model stable?” Yes. “Is the model doing what the business wants?” Never asked.

The Risk Framework You Actually Need

Real enterprise model risk has three layers:

Layer 1: Technical Risk. Is the model working as designed? This is what academic frameworks address. Data quality, concept drift, distribution shift, confidence calibration. These matter. Monitoring these is necessary but not sufficient.

Layer 2: Business Risk. Is the model’s decision-making aligned with business goals? This is where most frameworks stop. You need to actually validate that the model is making the business decisions you want it to make, not just making accurate predictions.

Layer 3: Governance Risk. Is someone explicitly accountable for what the model does? And if the model’s behavior conflicts with policy or regulation, do you have a way to know and a way to respond?

Here’s how you’d actually monitor a credit model:

Technical layer: Monitor data quality (are the features that go into the model staying the same?). Monitor performance (is accuracy stable?). Monitor distribution (are the inputs to the model changing in unexpected ways?).

Business layer: Monitor outcomes in context. If approval rates are changing, is that because credit quality changed, or because the model changed? If the model is approving more customers from some segments than others, is that intentional? What’s the approval rate spread you’re willing to accept?

Governance layer: Is there someone who’s reviewed the outcomes and said “yes, this is what we want”? If the model is declining business that was profitable, who made that decision? If the model’s risk tolerance is more conservative than the business wants, who’s changing which one?

Why This Actually Matters for Your Business

The reason this distinction is critical is that most enterprise models are making proxy decisions.

You don’t actually want a credit model that’s “accurate at predicting default.” You want a credit model that makes profitable lending decisions, recognizes systemic risk, and complies with regulation. Accuracy is a means to those ends, not the goal itself.

If your model is 95% accurate at predicting default but it’s also declining 30% of applications that would have been profitable, you have a problem. The academic framework will say “the model is accurate.” The business framework will say “the model is costing us money.”

In a lot of enterprises right now, this is exactly what’s happening. Models that were built to be accurate are getting used to make business decisions, and nobody’s checking whether the business decisions are good.

The technical team builds a model that predicts well on the training data. It passes validation. It gets deployed. The business starts using it. Six months later, someone notices it’s making different decisions than the human process it replaced, and nobody knows if that’s good or bad.

What Changes When You Separate These Risks

The first thing that changes is accountability. If a model is making bad business decisions, it’s not “a model accuracy problem.” It’s “a governance problem” or “a business decision problem.” And the person who’s accountable is not the data scientist. It’s the person who’s using the model to make decisions.

The second thing that changes is what you monitor. You don’t just look at technical metrics. You look at decision outcomes. You build feedback loops that tell you whether the model is doing what the business wants it to do.

The third thing that changes is escalation. If the model is making technically correct predictions but bad business decisions, you need a clear path to change the model, change the policy, or change the usage. And you need someone accountable for that decision.

The Conversation to Have Now

Most enterprises have started monitoring technical model risk. That’s good. But they haven’t started monitoring business model risk, and they definitely haven’t integrated that into governance.

The conversation you should be having is not: “Do we have good model monitoring?” It’s: “Do we know whether our models are making the business decisions we want them to make?”

That requires:

1. Clear business objectives for each model. Not “predict well.” What decision do we want the model to make? What constraints does it have? What risks are we accepting?

2. Outcome monitoring that’s connected to business metrics. If the model changes its behavior, do we know? If the decision outcomes change, is that intentional?

3. Escalation clarity. If the model’s decisions diverge from the business goals, who has authority to change the model or change the policy?

4. Explicit ownership. Not “the data science team owns the model.” Someone owns “we’re using this model to make X decision and it’s working as intended.”

This is not about building more sophisticated risk frameworks. It’s about connecting the technical risk work to the business decision that’s actually being made.

The Risk That Looks Like Success

The most dangerous scenario is the one where the model is technically sound, it’s making consistent decisions, it passes all the monitoring checks—and it’s making decisions the business doesn’t actually want.

You won’t notice this until it matters. Until someone asks, “Why did we decline that credit application?” and the answer is “the model said so,” and the question back is “but is that what we actually want?”

If you can’t answer that quickly and clearly, you have a model risk problem that no amount of academic monitoring will catch.

A financial services company shipped a credit decisioning model that started rejecting credit-worthy applicants from a specific zip code. When the incident got escalated, the conversation went like this:

“How did this happen?”

“The data was biased.”

“Why was the data biased?”

“The historical training data reflected past lending patterns.”

“Who owns the data?”

“The data team.”

So the data team got blamed. The data scientist who built the model got blamed. The incident was closed as a “data quality issue.”

But that’s not what happened. What happened was a business decision: to use historical lending data to train a credit model, knowing that the data contained biases from past discrimination. And nobody made that decision consciously. Nobody even knew it was a decision.

This is the accountability inversion. We blame the people who execute (the data team) and ignore the people who decided what to build (everybody else). And we do this systematically, in almost every company I’ve worked with.

How the Inversion Works

Here’s the mechanism. You have a process: data sourcing → data cleaning → model training → validation → deployment.

Someone (product, business, leadership) decides what problem to solve. Someone in product/engineering decides how to approach it. The data team gets requirements. They execute against those requirements. A model comes out. It gets shipped.

Something goes wrong.

Everyone looks back at the process. The data team was the last team that touched the model before it broke. So the data team gets blamed.

But the failure wasn’t in execution. The failure was in the decision about what to build.

If the requirements were: “Build a model using historical lending data to predict credit-worthiness,” and the data team said, “This data contains systemic bias,” and nobody responded by changing the requirements, then the person who ignored that flag made a decision. They decided to accept the risk of bias to get a faster model. That’s a legitimate decision. But it’s not the data team’s decision. It’s the business’s decision.

Instead, what happens is the data team raises the concern, the conversation stops, and when the model breaks, people say: “The data was biased.” As if bias is a natural property of data rather than a decision about which data to use.

The accountability inversion happens because people at the decision point don’t see it as a decision. They see it as “running the process.” The data team sees it as a constraint. And when something goes wrong, the constraint gets blamed.

Why This Matters More Than You Think

This matters because companies optimize based on where they assign blame.

If you blame the data team for bias, your company will hire better data scientists and invest in better data tools and build better data governance. These are all good things, but they don’t prevent the problem you actually had.

The problem you had was: someone built a model that discriminates against a protected class. That’s not a data problem. That’s a decision problem. That’s a product decision, a business decision, and a risk decision.

If you blame the data team, you’re saying: “The process failed because the execution wasn’t good enough.” So you make the execution better.

If you actually solved the problem, you’d be saying: “The decision-making was wrong. We made a conscious choice to use biased data and didn’t have the governance to catch it before shipping.” So you change the decision-making, not the execution.

And this inverts the incentives. The data team, knowing they’ll be blamed if something goes wrong, becomes more conservative. They push back harder on any requirements that seem risky. The business becomes more frustrated with data teams, because they slow everything down. The relationship becomes adversarial. And you end up with governance theater again.

Meanwhile, the actual decision-makers feel less pressure to think carefully about what they’re building. The worst outcome gets caught by the data team, so their job is to avoid shipping anything the data team flags. That’s a low bar.

What Actually Went Wrong

Let me walk through what should have happened in the credit model case.

Product says: “We want to predict credit-worthiness.”

Engineering/Data says: “We can do that. Here are the approaches: – Option A: Use historical lending data (fast, cheap, but contains systemic biases from past discrimination) – Option B: Use transactional data and alternative credit indicators (slower, more expensive, but less biased) – Option C: Use only recent lending data and accept that we have limited historical performance (somewhere in between)”

Then someone (product, business, risk, compliance) decides which option. That’s the decision. That’s the point where someone is making a choice about which risks to accept.

If you choose Option A, the decision is: “We’re accepting the risk of discriminatory outcomes in exchange for speed and cost.”

That’s a legitimate choice. You might make it. But it’s not a hidden choice. It’s an explicit decision that someone needs to own.

What usually happens instead is that the decision gets bundled into “let’s build the model” and nobody explicitly decides which option to take. The data team gets some vague requirements, picks the fastest path, and builds the model. When it breaks, you blame the execution.

The Three People Who Made the Decision

Here’s who actually decided to accept the bias risk:

1. The product person who framed the problem. If they framed it as “predict credit-worthiness from historical data,” they’ve already chosen Option A. They made a decision about data sources without realizing it.

2. The engineering/data person who translated the requirement into a concrete approach. They could have pushed back and said “this data is biased; here are the implications.” If they didn’t push back, they made a decision to accept it. If they did push back and were ignored, then the person who ignored them made the decision.

3. The business/leadership person who accepted the timeline and cost of Option A. If you decide your risk tolerance is “whatever the data allows” rather than “we need to actively manage bias,” that’s a decision with real implications.

Nobody is evil here. Nobody is incompetent. But three different people made pieces of a decision without any of them explicitly owning the whole decision. So when the model breaks, everyone points at someone else.

The data team gets blamed because they’re the last person in the chain. But they didn’t decide to use biased data. They were given a requirement and executed it.

What Accountability Actually Looks Like

Accountability means: someone explicitly decided and someone explicitly owns the outcome.

In the credit model case, here’s how it should work:

Before shipping: – Product frames the problem. (Decision 1: what are we solving?) – Product/Engineering present options with tradeoffs. (Transparency: here’s what each option actually means) – Someone (let’s call it Risk or Product leadership) decides which option. (Decision 2: which risk do we accept?) – Data/Engineering executes to that decision. (Execution with clear constraints)

If something goes wrong: – If the model breaks because of a data quality issue (execution failure), the Data team owns it. – If the model breaks because of a decision we made (the bias was predictable from the approach), the person who made the decision owns it.

The accountability framework clarifies who decided what. It doesn’t blame people for circumstances they didn’t choose. But it also doesn’t let decision-makers hide behind “the data was biased.”

The Rationalization That Kills Accountability

I’ve watched this happen in almost every company:

Something goes wrong. Someone asks: “Whose fault is this?”

The convenient answer is always “the process failed” or “the data was bad” or “the execution wasn’t good enough.”

The hard answer is: “We made a decision about what to build that we should have thought through more carefully.”

So companies invest in better processes, better data tools, better execution frameworks. And the same decision-making failures happen again, just with better process theater around them.

What changes when you flip the accountability is that decision-makers have to think about what they’re actually deciding. It’s not: “Do we want to move fast?” It’s: “If we move fast using this approach, what are the actual implications? Who’s the decision owner if those implications are bad?”

That conversation is uncomfortable. It’s easier to blame the data team.

The Pattern to Watch For

If your company has a pattern where the data team is blamed when ML models break, or where technical teams are blamed for bad outcomes, or where execution is blamed for strategy failures, you have an accountability inversion.

The fix isn’t better data governance or better technical execution (though those might help). The fix is clarity about who decided what.

Stop asking: “What went wrong with the execution?”

Start asking: “What decision was made about what to build, and who made it?”

And when you get that clear, you stop blaming the data team. You either blame the person who made the decision badly, or you stop treating it as a blame situation and start treating it as a learning situation.

Actually, you probably want to stop treating things as blame situations anyway. But first you have to stop blaming the wrong people.

I walked into a Fortune 500 company’s AI governance meeting and saw the org chart: 14 people, representatives from Legal, Compliance, Risk, Product, Engineering, Finance, and three layers of leadership. The meeting itself had run for six months with no shipping decisions made, but plenty of documentation produced. When I asked the head of the program what problem this committee was actually solving, she said: “We’re ensuring governance.”

That’s not governance. That’s the appearance of governance.

Governance is a decision-making framework. The best decision-making frameworks are not democratic. They’re not designed to include everyone. They’re designed to ensure the right people are involved at the right decision points, with clear authority, clear escalation paths, and clear consequences for being wrong.

The difference between these two things is the difference between a committee that produces theater and a governance structure that produces decisions.

How Committees Become Theater

Here’s what happens when you design governance through consensus:

You assemble people from different functions with different incentives. Finance wants to minimize risk and cost. Legal wants comprehensive documentation. Product wants speed. Engineering wants stability. They all get a vote.

The only outcome that keeps everyone happy is the one that offends everyone equally. So you set rules that are both conservative (Legal is satisfied) and bureaucratic (Finance is satisfied) and slow (Risk is satisfied) and documented (Compliance is satisfied). Everyone signs off. Nobody is happy. The system is designed not to make decisions, but to distribute responsibility so that when something goes wrong, you can point to the process instead.

The meetings get longer. More people get invited. Someone suggests a charter document. Someone else suggests subcommittees. Pretty soon you have a governance structure that exists to ensure it can explain why it didn’t make the decision anyone wanted.

I’ve watched this happen in companies with smart people making rational decisions. They’re not trying to build theater. They’re just following the logic of “if we want everyone to agree, we need to make decisions nobody disagrees with,” which means decisions that satisfy nobody.

The problem is that AI governance is not primarily a legal or compliance problem. It’s a speed and risk tradeoff. Committees are designed to minimize disagreement, not to optimize for speed or actually manage risk.

What Governance by Design Actually Looks Like

The companies that are handling this well have made a structural decision: they’ve separated the decision into parts.

Who decides what gets built: Product and Engineering. Speed is necessary here. The committee has no role. They have constraints (legal, compliance, risk), but constraints are not votes.

Who decides if those constraints are met: Legal, Compliance, Risk. Their job is not to vote. Their job is to flag when a proposal violates the constraints. If the constraint is “all customer data must be encrypted in transit,” then a proposal that doesn’t encrypt in transit gets sent back. The decision-maker has to either change the proposal or change the constraint (which escalates).

Who has authority to change constraints: Finance and executive leadership. This is the only place where you need consensus, because changing the constraints is actually changing the business model or the risk appetite. If the constraint is “all AI systems must use approved models,” and Product wants to use an unapproved model, that’s a constraint change. That’s the conversation for leadership.

Who owns the outcome: Product and Engineering, with escalation to leadership if constraints are violated. This is critical. The people who make the decision have to own the consequences. If they ship something that violates compliance, it’s on them, not on “the committee.”

This structure has a critical property: it makes decisions fast because there’s no voting. It’s also clear about what’s being constrained and why. And it’s clear who’s accountable.

Why This Actually Reduces Risk Better Than Committees Do

Here’s the counterintuitive part: decentralized governance with clear ownership actually manages risk better than consensus governance.

When everyone is responsible, nobody is. The committee that approved something problematic can always point to the process. But when Product and Engineering own the decision, and they know that Legal will flag constraint violations, and they know that constraint violations escalate, they think more carefully about what they’re shipping.

Also, a committee of 14 people will never move as fast as a product team. So you actually get worse outcomes: slower decisions and less careful consideration because the decision-making is so slow that nobody’s paying attention anymore.

The framework I’m describing isn’t “let engineering do whatever they want.” It’s “engineering makes the call, but within clearly defined constraints, with clear escalation paths when those constraints conflict.” That’s radically different from consensus.

The Conversation About What You’re Actually Trying to Prevent

Here’s where most companies go wrong. They build a governance committee without first deciding what they’re actually trying to prevent.

Are you trying to prevent: – Systems that break laws (you need Legal with veto authority, but that’s narrow) – Systems that harm customers (you need Product with clear accountability) – Systems that lose money (you need Finance with clear cost constraints) – Systems that offend people (this is a different problem; it’s about policy, not governance) – Systems that break architecturally (you need Engineering with clear technical standards)

The answer matters because different things need different governance structures. If you’re trying to prevent legal violations, you need a clear compliance rule and someone with authority to enforce it. If you’re trying to prevent customer harm, you need clear product standards and someone accountable for them. If you’re trying to prevent everything, you’ve got a committee.

Most enterprises build committees because they’re trying to prevent everything, and they haven’t had the conversation about what the actual constraints are. So they generate rules to cover every possible scenario, which means nothing ships, which means the governance structure prevents you from building AI at all.

What Actually Changes When You Design for Decision-Making

The shift from committee to designed governance requires:

1. Clarity about constraints, not votes. What are the actual constraints? Make them explicit. “All AI systems must use approved models” is a constraint. “The committee must agree” is not. Constraints are narrower and measurable.

2. Clear escalation paths. When a constraint conflicts with a business goal, where does that decision go? Not “back to the committee.” To a specific person or leadership group with actual authority to change the constraint.

3. Accountability that sticks. If Engineering ships something that violates a constraint, it’s on them (with escalation to leadership if it’s a material risk). This makes them think carefully about what they’re shipping.

4. Separation of concerns. The people who decide if you build something (Product) are different from the people who decide if you can build it that way (Legal/Compliance). Both groups are necessary. Neither has a vote.

This requires culture shift. Companies have been trained to believe that shared decision-making is fair, and that consensus is legitimate. Neither is necessarily true. Clear authority with clear accountability is fairer and faster than democratic committees that don’t actually decide anything.

The Governance Structure No One Talks About

Here’s what I never see in organizational documents: “The AI Governance Committee will not make decisions. Its purpose is to flag constraints and escalate conflicts. The decision authority rests with [specific role].”

That sentence would solve most governance theater problems. But it requires a company to say clearly: “Product owns the decision. Legal flags constraint violations. Engineering owns the technical quality. Finance owns the cost constraints. Leadership breaks ties.”

The theater happens when you try to maintain the fiction that everyone is equally responsible for the decision. The real governance happens when you distribute responsibility clearly and make someone actually accountable.

Your AIOC might be fine. But if you can’t answer the question “who actually decides if we build this?” in less than a sentence, it’s probably theater.

A CTO told me they’re building their own fine-tuned version of an open-source model to reduce vendor dependency.

I asked: Why?

The answer was: “So we’re not locked into one vendor.”

That’s the right concern. The execution is usually backwards.

Building your own fine-tuned model doesn’t make you independent. It makes you dependent on a different thing: your ability to maintain it. And that’s a dependency most enterprises underestimate.

Independence Isn’t About Which Model You Use

Strategic independence in AI is a real problem. I’ve written about it already (vendor dependency, lack of abstraction layers). But what independence actually requires is often misunderstood.

Most organizations think independence means: “Don’t use foundation models. Build or fine-tune your own.”

That’s treating the symptom, not the disease.

The disease is structural dependence on any single infrastructure or capability that you can’t replicate if the vendor relationship changes. The cure is designing your systems so that core capabilities either: a) exist across your organization as institutional knowledge, b) can be replaced without rebuilding the entire application, or c) are decoupled from your core business logic.

Building your own fine-tuned model can accomplish this. But so can building abstraction layers. So can designing for model rotation. So can maintaining real technical depth in how your models work.

The question isn’t “should we build or buy.” The question is “what do we need to maintain control over, and how do we actually maintain it?”

What Independence Actually Costs

Let me be concrete about what it means to build and maintain your own fine-tuned models.

First: initial build cost. You need engineers who understand model training, which means data scientists with 5+ years of experience. If you need to build from scratch (not fine-tune), you’re talking $1-3M in development cost, 6-12 months of timeline, and a team of 4-8 people.

Second: ongoing compute costs. A foundation model API costs you based on usage. Running your own model costs you compute (GPU infrastructure, maybe $100K-500K annually depending on scale) plus the electricity, networking, and datacenter costs that come with that.

Third: maintenance burden. When a new model comes out that’s better, should you retrain? When your domain shifts and your training data becomes stale, should you retrain? When you discover bias in your model, how do you fix it? Each of these is an active maintenance project, requiring someone to be responsible for it.

Fourth: talent concentration. You now have a capability that only a few people in your organization understand deeply. If your model expert leaves, or is overloaded, or moves to a different team, you have a problem. You’ve traded vendor dependency for person dependency.

Fifth: opportunity cost. While you’re building and maintaining your own model infrastructure, you’re not building the applications that use it. You’re spending engineering time on infrastructure that, for most organizations, a vendor could provide more efficiently.

The argument for building your own typically goes: “This gives us independence.”

The reality usually is: “This trades one dependency for another, often a more expensive one, and concentrates risk in a different place.”

When Building Your Own Actually Makes Sense

There are scenarios where building your own model is the right answer:

Scenario 1: Proprietary advantage through data. You have data that nobody else has access to, and that data gives you competitive advantage. You’re not just using a foundation model as a starting point. You’re incorporating proprietary data to create a capability that competitors can’t easily replicate. The privacy and competitive protection from keeping this internal is worth the build cost.

Example: A financial services firm with 50 years of internal transaction history that becomes part of the model. A manufacturing firm with sensor data from their equipment that’s unique to their process.

This is a real reason. But notice: you still might use a foundation model as a base and fine-tune it with your proprietary data. You don’t necessarily need to train from scratch.

Scenario 2: Extreme scale or cost sensitivity. You’re running this model so many times that the marginal cost of API calls is expensive relative to running your own infrastructure. You’ve done the math, and your compute costs are 30% lower running your own.

Example: A company processing millions of requests per day where each API call costs money, and running your own infrastructure has better economics.

This is a real reason. But notice: you still need the expertise to maintain the infrastructure. And you need to be comfortable with the operational burden.

Scenario 3: Regulatory or data residency requirements. Your industry requires that models be trained on data that never leaves your infrastructure. You can’t send data to a third-party API. You need the model to run in your environment.

Example: A healthcare provider with patient data, a financial institution with transaction data, a government agency with classified data.

This is a real reason, and it’s absolute. You have no choice. But notice: you might still use a foundation model as a base (open-source, self-hosted, or fine-tuned on your infrastructure) and then add your domain-specific fine-tuning.

In all three scenarios, the justification is about data (proprietary, sensitive, scale-related). Not about “independence from vendors.”

What Independence Actually Requires (If You’re Not Building Your Own)

If you’re not building your own model, but you still want strategic independence, here’s what actually works:

First: Design abstraction layers that are model-agnostic.

Instead of writing code against a specific model vendor’s API, write code against an abstraction. Different models can satisfy that abstraction. It’s more complex. It’s less efficient (you can’t optimize for vendor-specific quirks). But if your primary model changes, you can migrate without rebuilding.

This is the most practical approach for most organizations. It gives you real independence without the cost and complexity of building your own. This applies whether you’re building on any cloud AI platform, open-source models, or proprietary APIs—the principle is architectural, not about any specific vendor.

Second: Maintain real technical depth about how your models work.

Don’t treat models as black boxes. Understand what they’re doing, why they work, what they fail at. This requires people who can debug models, understand prompt behavior, interpret outputs. If you have people who really understand models deeply, you’re not dependent on any single vendor because you have the skill to work with alternatives.

Third: Build model rotation into your roadmap.

Plan for changing models not as a crisis scenario, but as a regular architectural activity. Every 18-24 months, do an evaluation of foundation models. Test alternatives. Understand switching costs. Keep up relationships with multiple vendors. When you do need to switch, you’re not starting from zero.

Fourth: Keep the model separate from your application logic.

The model is a component, not the core. Your application has business logic that’s independent of which foundation model it’s using today or tomorrow. The model is pluggable. This architecture costs more upfront but gives you real flexibility later.

Fifth: Maintain financial optionality.

Budget for alternatives. Set aside 5-10% of your AI budget for evaluating options. Build organizational practices where model choice is a strategic decision, not a habit. If your only model is because it’s the one you started with, that’s dependency. If your primary model is chosen and re-evaluated regularly, that’s strategy.

Why This Matters For Your Board

Strategic independence in AI infrastructure will matter more as enterprise AI becomes more central to business operations. Right now, most organizations are still experimenting. Dependency feels theoretical.

Over the next two years, as AI systems become embedded in core business processes (customer service, decision-making, optimization), a vendor change becomes a significant operational event. Your systems, your teams, your customer-facing capabilities depend on that vendor’s continued cooperation and reasonable terms.

The organizations that will have optionality are the ones that designed for it from the start. Not the ones that panic later and decide to “build independence” by rewriting their infrastructure from scratch.

Building that independence doesn’t require building your own models. It requires:

• Architecture decisions that allow model swapping
• Technical depth so you understand models, not just use them
• Regular evaluation of alternatives so you don’t wake up dependent
• Clear separation between model and application
• A serious answer to “what would we do if this vendor changed terms?”

The organizations that get this right won’t necessarily switch models. They’ll just have the ability to, which means they’ll negotiate better, maintain strategic clarity, and adapt faster when the foundation underneath them shifts.

Independence is possible. It just doesn’t come from building your own model. It comes from designing your systems as if you might need to change it.

You built something good. The model works. The infrastructure is solid. You’ve got buy-in from the business sponsor. You’ve trained the team. You’re ready to go.

Six months later, 40% of the team is using it. Eighteen months later, it’s been deprecated.

This isn’t a technical problem. It’s not a governance problem. It’s an adoption problem that looks like a technical problem, and most organizations don’t see the pattern until it’s too late.

What Most Organizations Get Wrong About Adoption

The standard narrative about adoption goes like this: “We built a tool. People didn’t want to use it. Maybe the tool wasn’t good enough, or the training wasn’t good enough, or the business case wasn’t compelling enough.”

And then the organization either: a) doubles down on training, b) adds more features to make it more compelling, or c) gives up and builds something else.

None of these is wrong exactly. But they’re all treating adoption as primarily a tool or training problem. It’s not.

Adoption in organizations isn’t like adoption in consumer markets. In consumer markets, if you build something better than alternatives, people use it. They compare options and pick. They vote with their usage.

In enterprises, adoption is a work context problem. People don’t choose what to use based on whether it’s best. They choose based on: Does this fit into how I already work? Does using this create friction with other parts of my job? Does my manager expect me to use this? Do the people I work with use it?

The best tool in the world will fail if it doesn’t fit into the actual context of how the work happens.

The Real Adoption Curve (Not The One In The Literature)

Most adoption literature talks about an S-curve: early adopters, early majority, late majority, laggards. You get 10% adoption, then 20%, then it accelerates. Classic diffusion of innovation.

That’s not what happens with enterprise tools.

What happens with enterprise tools is:

Month 0-2 (Honeymoon): High enthusiasm. The team that built it is excited. The sponsor is excited. Early adopters are excited. You see 50-60% usage among people who could theoretically use it. People are trying it. Everyone’s optimistic.

Month 3-4 (Friction Discovery): Usage starts to flatten. You’ve got 50% usage, and it’s not growing. The people who are using it are hitting edge cases. The tool doesn’t work perfectly for their specific workflow. You ask why usage isn’t higher, and you get feedback like: “It doesn’t integrate with my existing process.” “It’s faster for me to do it the old way and copy the result in.” “I can’t use this when I’m at a client site.” “The output isn’t in the format I need for my downstream system.”

None of these are “the tool doesn’t work.” They’re “the tool doesn’t work in my context.”

Month 5-6 (The Push-Back): Leadership decides usage is too low. They mandate using the tool. Or they add requirements: “You have to use this by the end of Q2.” Or they tie it to performance reviews: “Usage of this tool is part of your evaluation.”

This is when you learn whether you’ve actually solved the context problem or just built a nice tool.

Month 7-12 (Deterioration): If you fixed the context problems, usage stabilizes at a higher level. If you didn’t, usage drops below the mandate level as soon as the mandate is lifted. People find workarounds. They convince themselves the tool is “not quite right for my situation” so they go back to the old way. They batch the work they have to do with the tool with work they do the old way, then copy the results in. Adoption grinds down.

Month 13+ (Abandonment): The tool is still there. Some people use it. But it’s no longer the default. New people aren’t trained on it because “it didn’t really take off.” The organization moves on to the next initiative.

The standard response to this pattern is: “The tool wasn’t good enough.” The actual response should be: “We didn’t understand the context well enough.”

What Context Actually Means

When I say “context,” I don’t mean “people don’t like change.” I mean the specific, structural constraints on how work actually happens.

Say you built an AI model that improves sales forecasting. It’s a good model. Salespeople who use it get better at predicting. But your organization has a specific context:

• Salespeople are evaluated monthly on their forecast accuracy
• They’re compensated based on hitting their forecast
• The model updates twice a week
• But they submit their forecast once a week
• And once submitted, changing the forecast requires approval from their manager
• Who is traveling this week

In this context, the salesperson faces an explicit cost to using your better model: they have to get approval to change a forecast they’ve already submitted. The benefit (better accuracy) is diffuse and doesn’t show up in their evaluation immediately. So they don’t use it.

This isn’t a training problem. This isn’t a “people resist change” problem. This is a structural misalignment between when the tool is useful and when the person is incentivized to use it.

The way you fix this isn’t by training harder. It’s by understanding the structural context and redesigning around it. Maybe the model needs to run on the person’s schedule, not its own. Maybe the forecast needs to be re-submittable without approval. Maybe the evaluation system needs to change. Maybe you need to give people a 30-day look-ahead so they can submit more aggressive forecasts and let the model refine them.

None of this is about the model. It’s all about the work context.

Three Patterns That Kill Adoption

I’ve watched three adoption-killing context problems show up repeatedly:

1. Workflow Mismatch

You built a system that works great if people follow a certain workflow. But that workflow doesn’t match how people actually work.

Example: You built a data preparation tool that works great if people upload data, review it, and run preprocessing. But people’s actual workflow is: I’ve got data in a database, I need to run analysis in a spreadsheet, and I need results in a presentation. Your tool is in the middle of their workflow but doesn’t integrate to the beginning or the end. They have to export from the database to the tool, import the results into the spreadsheet, then copy the numbers into the presentation. That’s three tools instead of two, so they don’t use yours.

2. Incentive Misalignment

You built a system that benefits the organization, but the person using it doesn’t benefit individually.

Example: You built an AI system that optimizes staffing across departments. It’s great for the organization. But a department manager’s bonus is based on their local efficiency, not global efficiency. Using your system might move one of their people to another department, which makes them look bad locally. So they don’t use it even though it’s better for the company.

3. Authority or Credibility Gap

You built a system that provides recommendations or guidance, but the person using it doesn’t trust the system or its recommendations.

Example: You built a model that recommends treatment options for doctors. The model is accurate. But doctors are trained to make their own clinical judgments. They don’t trust a model’s output more than their own experience. Unless you can demonstrate that the model is more accurate than they are—which requires time and data they don’t have—they won’t trust it enough to use it over their existing judgment.

Each of these patterns shows up differently in different contexts, but they all have the same underlying structure: the tool is good, but using it costs the person something (time, authority, credibility, incentive alignment) that makes it rational not to use it.

How To Actually Fix Adoption

The organizations that get adoption right don’t start by building the model. They start by understanding the context.

Map the workflow. How do people actually do this work today? Don’t ask them—watch them. Or better, do the work with them for a few days. You’ll see things they don’t even notice they’re doing.

Identify friction points. Where in the workflow would your tool create extra steps? Where would it require people to change how they work? Where would it require approval or credibility they don’t have yet?

Understand incentives. Is the person who will use this tool evaluated on the thing your tool helps with? Or are they evaluated on something else? If they’re evaluated on something else, using your tool is a cost to them, not a benefit.

Design for the context, not against it. Build the tool to fit into the workflow people actually use, not the workflow you wish they used. If people need results in spreadsheets, make sure your tool outputs to spreadsheets natively. If people are evaluated on local metrics, make sure your tool makes them look good locally, even if it’s making global trade-offs.

Plan for the adoption curve. Don’t expect the S-curve. Expect the friction curve: high, then flat, then mandated, then deteriorating. Plan for each phase. Build in the changes you need to make to get past the friction phase before you deploy.

Why This Matters

The cost of abandoned tools is higher than you think. You spent development time. You spent infrastructure costs. You spent training time. You spent organizational attention. And then six months later, it’s not used.

But the bigger cost is that the organization loses confidence in your AI program. One abandoned AI tool creates skepticism about the next one. The team that built it loses credibility. The sponsor becomes hesitant. By the time you want to try again, you’ve burned through social capital.

The organizations that are winning with AI aren’t the ones building the most sophisticated models. They’re the ones that build models that fit into how people actually work, and then systematically manage the adoption curve until the tool becomes part of the standard workflow.

Adoption isn’t something that happens if you build it right. It’s something you design for, plan for, and manage through the friction phases.

A VP of Engineering told me about their AI project last year. Budget was $2M. Actual spend was $6.8M. When I asked what happened, she said: “We didn’t understand what we didn’t understand.”

That’s not a failure of planning. That’s a failure of the budgeting framework itself.

The organizations I’ve seen blow through AI budgets aren’t usually incompetent. They’re using budgeting frameworks that were built for a completely different kind of work. And they don’t realize it until six months in, when the budget is half-spent and the project is 20% complete.

Why Traditional Project Budgeting Fails For AI

Traditional software project budgeting assumes a few things: – You know what you’re building – You know roughly how long it will take to build – You know what “done” looks like – You know what success means

AI projects violate every one of these assumptions.

You think you’re building a recommendation system. Once you start, you realize your data quality isn’t good enough. You spend six weeks fixing data. Then you realize your features don’t predict what matters. You spend another eight weeks rebuilding features. Then the model works, but it produces biased outputs. You spend another month on fairness improvements. None of these were “surprises” in the sense that they were unknown risks. They were surprises in the sense that you didn’t know they were going to be the blocking issues.

The fundamental problem is that AI projects are exploration work disguised as execution work.

In traditional software, you explore during the design phase. You do architecture reviews. You write specs. You plan. Then you build. The build phase is well-understood. You have precedent. You have patterns. You have experienced people who know how long it takes.

In AI projects, the exploration doesn’t stop. You’re executing, and you’re discovering simultaneously. You’re building the model and learning what the model can do. You’re testing it and discovering what it fails at. You’re deploying it and discovering what real-world performance looks like. Each phase reveals information that changes the phase that comes next.

So you budget for the plan you made in month one. But month three reveals that the plan was 40% wrong. Do you stop and replan? Most organizations don’t. They replan and continue. Which means you’re now $800K into a $2M budget, and you’ve got $1.2M left to finish work that actually costs $1.8M based on what you know now.

Where The Costs Actually Hide

There are five categories of AI project costs that traditional budgeting frameworks don’t account for:

1. The Data Prep and Exploration Tax

You need training data. In traditional software, data is often already there. In AI, you often need to acquire, clean, label, and prepare data. A team of people might spend 40% of the project doing this. And you don’t really know how much you need until you build the first model and discover what’s missing.

One organization budgeted $200K for data prep. Actual spend: $800K. Why? They discovered their labels were inconsistent. They had to relabel a dataset of 100K examples. They discovered they needed geographic variation they didn’t have. They had to acquire new data. They discovered the time-window for some features was wrong. They had to rebuild the dataset. This isn’t incompetence. This is the normal cost of working with data when you don’t fully understand it upfront.

2. The Model Exploration and Iteration Tax

You build a model. It gets 87% accuracy. Is that good? You don’t know without a baseline. You don’t know without understanding what 87% accuracy means for your business. Does a model that’s 87% accurate but biased against a particular group meet your requirements? You don’t know until you measure it.

So you iterate. You try different architectures. You try different feature sets. You try different training approaches. Each iteration takes computational time and engineer time. A team might do 30-40 iterations before landing on something that’s both technically sound and meets business requirements.

Budget for this? Most organizations don’t. They budget for “build the model.” They don’t budget for “understand the model well enough to deploy it.”

3. The Validation and Governance Overhead

Before you deploy, you need to validate that the model does what you think it does. You need to test it against edge cases. You need to check it for bias. You need to make sure it handles distribution shift. You need to document the failure modes.

This is where governance overhead becomes a cost item, not a process item. A data scientist might spend 30% of their time building the model and 70% of their time validating, documenting, and preparing for governance review.

Most organizations budget for “build” and assume “validate” is 10-20% overhead. In practice, it’s often 50-70% overhead, and it compounds when the governance structure isn’t clear (which it usually isn’t on first AI projects).

4. The Integration and Infrastructure Overhead

The model doesn’t live in isolation. It lives in your product. To make that work, you need to: – Set up infrastructure to serve the model – Build monitoring to track model behavior – Set up logging to understand what the model is doing – Build feedback loops to collect data on what’s actually happening vs. what you predicted – Set up retraining pipelines so the model stays fresh – Build fallback systems in case the model fails

This is often 30-50% of the total project cost, and it’s rarely budgeted separately. It gets rolled into “engineering overhead” or “platform overhead.” Then halfway through the project, the infrastructure work surfaces as a separate project that wasn’t in the original plan.

5. The Expectation and Iteration Tax

You deliver the model. It works. But the business stakeholders have ideas. What if we changed the optimization target? What if we added this constraint? What if we included this data source?

Each of these is reasonable. Each is a couple weeks of work. But if you’ve got three business stakeholders with two ideas each, that’s six iterations that weren’t in the original plan.

This is particularly tricky because it’s not the AI team’s fault. It’s the normal cost of stakeholder collaboration. But it’s not in the budget.

How Projects Actually Fail (It’s Not What You Think)

Projects blow up not because the AI is hard. Projects blow up because one of these five cost categories explodes, and there’s no buffer.

You start with: – $500K for team salaries (data scientists, engineers) – $500K for computing costs – $300K for infrastructure – $700K for contingency

Then you discover in month four that your data quality problem requires reworking 60% of your dataset. That’s $200-300K of work that wasn’t in the “data prep” budget. You absorb it from contingency.

Then you discover in month six that the model architecture you chose doesn’t scale the way you expected. You need to rewrite the serving infrastructure. That’s another $150-200K. Contingency again.

Then your business stakeholder wants to optimize for a different metric because the business priorities shifted. That’s another $100-150K of model work and validation.

By month eight, your contingency is gone, your data team is burned out because they’ve been fighting data issues for six months, your infrastructure team is frustrated because they’re building stuff that’s changing based on model decisions, and you’ve got $500K left and need $800K more to finish.

The project doesn’t fail because AI is hard. It fails because the budgeting framework assumed these categories of work were known and fixed, when they’re actually variable and interdependent.

What Budgeting Actually Needs To Account For

If you’re going to budget for an AI project correctly, you need to plan for:

1. A discovery phase with explicit budget. Separate from the build phase. If you’re going to learn 40% of what you need to know from building, budget that as learning, not execution. This might be 2-3 months and $300-400K for a medium-sized project. The output is not a production model. It’s clarity on what the production model will require.

2. Variable data and exploration costs. Budget for data work separately and generously. Assume you’ll need to rework data. Assume you’ll need more data than you think. Assume you’ll iterate on features and labeling. A 40% buffer on data costs is normal, not unusual.

3. Governance and validation as a distinct work stream, with separate ownership. Not a 10% overhead tax. A separate team, separate timeline, separate budget. If you’ve got a data scientist building a model, you need a governance architect validating it. That’s a different resource, different cost.

4. Infrastructure and serving as a separate project. The model is not the product. The system that makes the model production-ready is. Budget these separately. Assume infrastructure will be 30-50% of the total project cost if you’re building new.

5. Explicit contingency for stakeholder iteration. If your business doesn’t know what success looks like, budget for them to figure it out while you’re building. That might be 1-2 iterations that weren’t in the spec. Budget them.

6. Explicit assumptions about model performance. Don’t budget based on “build a model.” Budget based on “build a model that achieves X accuracy on Y metric while satisfying Z constraints.” The constraints matter more than you think, and they cost real money to validate.

The Math That Actually Works

Here’s what a real budget might look like for a $2M AI project:

• Discovery phase: $300K, 2-3 months. Goal: understand data, validate problem, identify architecture approach.
• Build phase: $800K, 4-5 months. Build model, do initial validation, iterate on architecture.
• Governance and validation: $300K, 2-3 months. Deep validation, bias testing, failure mode analysis, documentation.
• Infrastructure and serving: $400K, 3-4 months. Build serving infrastructure, monitoring, retraining pipelines, integration.
• Iteration and contingency: $200K, ongoing. Handle stakeholder changes, unexpected issues, performance improvements.

Notice: the “model” is maybe 40% of the cost. Everything else is making the model real and acceptable.

Notice: governance is explicit, not overhead.

Notice: there’s actual contingency, and it’s for things you expect to happen.

This is a better framework not because it predicts perfectly. It’s better because it accounts for the actual sources of cost in AI projects, and it gives you room to manage them when they happen.

Your next AI project will blow up. Some part of it will cost more than you expected. The question is whether you budgeted for that or whether you pretended it wouldn’t happen.

Every large organization I’ve worked with has run the same hiring search: “Looking for an AI/ML Leader with 10+ years of machine learning experience, deep understanding of governance, strong executive presence, ability to translate technical concepts for boards, proven track record scaling data science teams, and experience in regulated industries.”

They’re looking for a unicorn. And it’s sinking their programs.

The person who is genuinely world-class at machine learning research isn’t going to want to spend 40% of their time in governance meetings. The person who is genuinely world-class at organizational design and governance isn’t going to have a publication record in top machine learning conferences. The person who is genuinely world-class at executive communication isn’t the same person who thinks deeply about model architecture.

You don’t need one person to be all three. You need a different organizational structure.

What Companies Actually Need (But Don’t Know How to Ask For)

Let me separate the actual functions that need to exist:

Function 1: Model Builder. Someone who understands how to train, fine-tune, or integrate foundation models. Understands technical tradeoffs, knows how to debug model behavior, can estimate computational requirements and inference costs. This person is a technologist. They might have a PhD in ML. They might have 5 years of production AI experience. They might have started last year and be brilliant. The key trait: they know AI systems can be built and how to actually build them.

Function 2: Program Governance Architect. Someone who understands how AI systems fail in organizational contexts, how to build accountability structures, what risks matter vs. don’t, how to translate technical capabilities into business constraints, how to design approval processes that actually work. This person might not be able to build a model. But they understand organizational design, they’ve worked in regulated industries, they’ve seen programs fail and know why. They’re probably not an ML researcher. They’re probably someone with operating experience and a deep understanding of governance.

Function 3: Executive Translator. Someone who can articulate what’s possible, what’s risky, what’s strategic about AI to boards and CFOs. This person might not understand backpropagation. They might not care about model architecture. But they understand business impact, board dynamics, shareholder communication, how decisions actually get made in the C-suite. They know how to translate “the model drifted” into “our margin impact is X.”

What organizations usually do is try to hire one person for all three. The result is: you hire someone who’s adequate at all three, great at none.

Why The Unicorn Hire Fails

The unicorn hire feels safer than the distributed model because there’s one person who “owns” AI. One P&L. One budget. One accountability line. Your board likes this. Your CFO likes this. It’s clean.

It fails for three reasons.

First: you’ve created a single point of failure. If the person leaves, or underperforms, the entire program slows or stops. You haven’t actually built organizational capability. You’ve hired individual capability. And individuals leave.

Second: you’ve created role confusion. The person is trying to do three jobs. They prioritize the one they like or are good at and deprioritize the others. Usually they deprioritize governance. Then you’re shocked to discover 18 months into your program that you have zero governance structure. This isn’t because they’re bad at governance. It’s because they’re spending 70% of their time building models and 20% of their time talking to executives. Governance gets 10%.

Third: you’ve hired against the market you actually face. The market doesn’t have people who are world-class at model building and organizational design and executive communication. So you hire someone who’s medium at all three. Six months in, you discover they’re not technically deep enough for your hardest problems, or not organizationally sophisticated enough for your actual governance needs, or not credible enough at the executive level. You’ve filled the role without solving any of the underlying problems.

What Actually Works: The Supporting Structure

The organizations that get their AI programs right have a different structure. It’s not one leader. It’s typically three people or roles working in explicit collaboration:

The Chief AI Officer or AI Program Lead is the integration point. They own the P&L, they own the roadmap, they manage the budget. But they don’t try to be the world’s best at everything. They’re typically 40-50% governance architect, 30-40% operator/executor, and 20% translator. They’re comfortable with not being the smartest person in the room on pure ML.

The Technical Leader (whether that’s a VP of Data Science, Head of Applied AI, or Principal Architect) is responsible for the actual AI/ML execution. They own the model builds, the infrastructure, the performance. They’re not trying to be a board-level communicator. They’re not designing organizational structures. They’re focusing on technical excellence and making sure the model systems work.

The Governance / Risk Lead is responsible for designing the structures, processes, and accountability mechanisms. They might sit in the CAIO’s organization, or they might sit in Risk/Compliance/Legal. But they’re explicitly responsible for governance design, not just “making sure we follow the rules.”

And there’s a Chief Executive / CFO / Board Member who is explicitly responsible for asking hard questions, making sure these three are actually collaborating, and understanding what the actual risks and opportunities are.

Notice what this structure has that the unicorn hire doesn’t: explicit accountability for each domain, no single person trying to be great at everything, and organizational incentives for collaboration rather than role confusion.

How to Actually Hire For This (If You Want To Start Now)

If you’re going to build this structure, the hiring conversation is different.

For the Chief AI Officer, you’re not looking for “deep ML expertise.” You’re looking for someone who understands organizational design, has operated in scaled environments, can read a balance sheet and understand how to translate technical decisions into financial impact, and can manage a diverse team. You probably want someone with 10+ years of operating experience. They might have never trained a model. That’s fine.

For the Technical Leader, you’re looking for someone who understands AI/ML systems and can ship them. Deep expertise in your specific domain is nice but not required. You want someone who can articulate technical tradeoffs, who has hands-on building experience (not just theoretical), and who can mentor the team. You probably want 5-8 years minimum, but that’s experience doing something, not background.

For the Governance Lead, you’re looking for someone who understands organizational risk, processes, and compliance. They might come from traditional software governance, or from regulatory work, or from program management. They need to understand what it means for an approval process to actually work. They don’t need to understand backprop.

And you need to explicitly structure their collaboration. Quarterly sync. Shared OKRs. Regular forums where disagreements get surfaced and resolved. Not friendship. Collaboration.

Why This Matters For Your Board

The reason this matters is that it changes how you talk about AI risk and opportunity. With the unicorn model, you’re always one person away from “what happens to our AI program?” With the distributed model, you have institutional capability.

It also changes how you think about scale. One person can manage one program. Multiple people can scale across multiple programs, multiple business units, multiple risk domains.

And it changes how you assess whether you’re actually solving the governance problem. With the unicorn, you can’t tell. With the distributed model, you can see which function is working and which is breaking.

This isn’t organizational theory. It’s practical. The organizations that have figured this out are the ones whose AI programs are actually running, generating value, and not creating constant crises.