A CTO I know recently described a conversation with their board: “We’ve built our entire competitive advantage on a foundation model API. What happens when your vendor changes their pricing? What happens when they change their terms? What happens when they decide to compete with us directly?”

The board had no answers. Not because they’re unprepared. But because nobody had modeled the scenario. It felt theoretical, or distant, or like someone else’s problem.

It’s not. It’s the central architecture problem facing enterprises right now, and most companies haven’t thought through the implications.

The Vendor Lock-in Is Structural, Not Accidental

Let’s be precise about what’s happened in the last 18 months. Foundation models have become genuinely useful, genuinely reliable, and genuinely easy to integrate into business systems. A team of three engineers can build a production system that was impossible to build in 2022.

That’s the good news.

The dependency is also structural, not accidental. You’re not using a foundation model as one option among five. You’re building on top of it because:

1. The capability is native to the model. You can’t replicate this behavior with a smaller model or open-source alternative because the few-shot reasoning, the instruction-following, the broad knowledge cutoff—these are unique to models of this scale.

2. Your domain adaptations are optimized for that specific model. You’ve tuned prompts for its strengths. You’ve built workflows that depend on its specific output format. You’ve designed your architecture around its API constraints. Moving would require rearchitecting.

3. Your team has built muscle memory. Your engineers think in terms of tokens. Your product managers understand the API’s quirks. Your operations team has learned how to handle the cost profile. There’s switching cost beyond the code.

When you’ve integrated this deeply, it stops being “we can swap vendors if we need to” and becomes “this vendor is our foundation.”

The question that should terrify you isn’t hypothetical: What’s your actual downside if the terms change?

What “Terms Change” Actually Looks Like

Let me name some scenarios that have already happened:

A foundation model provider raised pricing on their flagship API, and companies that had modeled economics around older versions had to reprice their products or absorb margin compression.

A foundation model provider changed their API rate limits, and companies hit unexpected bottlenecks in their production systems.

A foundation model provider changed their token-counting methodology, and billing shifted for systems that hadn’t anticipated the change.

A foundation model provider started offering cheaper models with similar capabilities, and companies that had built competitive advantages on the assumption of exclusive access to high-quality reasoning were suddenly competing on cost instead.

A foundation model provider changed their pricing on advanced context window features, and companies had to rebuild systems they’d optimized for that specific cost structure.

The pattern is consistent: these aren’t conspiracy theories or paranoia. These are normal business decisions from vendors optimizing for revenue and managing demand. And every time they happen, they force expensive recalibration on dependent companies.

But the scenario that should worry you more is the one that hasn’t happened yet: what if your foundation model provider decides to build a competing product?

The Unspoken Competitive Exposure

Here’s the asymmetry that matters: You’re optimizing your systems on their infrastructure. Your customer relationships, your domain expertise, your operational knowledge—these all live on top of their API.

They see your usage patterns. They see what you’re building. They understand your market. They have the infrastructure. They have the customer relationships.

If they decide your market is valuable, they can build a competing product faster, cheaper, and with built-in advantages (integration with their foundation model, direct access to their API, no external dependencies, direct customer contact for switching). You’ve essentially market-tested the opportunity for them.

This isn’t hypothetical either. It’s happened in cloud computing (major cloud providers started with foundational services but moved up into competitive applications). It’s happened in search (dominant platforms built ad networks, then started competing with the services that were built on top of search traffic). It’s the normal trajectory of platform companies.

The difference here is that the switching costs work in the vendor’s favor. If you’ve built an integration layer for one vendor’s API, moving to another foundation model provider isn’t just an engineering project. It’s a rebuild. They know this. You should price this risk into your strategy.

What Most Companies Are Actually Doing (And Why It’s Insufficient)

I’ve seen three patterns in how enterprises are thinking about this.

The Hedging Approach: “We’ll use multiple foundation models.” This reduces the risk that any single vendor’s downtime breaks your system. It doesn’t reduce dependency. You’re now dependent on multiple vendors, each of whom has independent incentive to compete with you or change their terms. It also increases operational complexity and cost. You’re not building redundancy; you’re building technical debt.

The Wait-and-See Approach: “We’re starting with one foundation model, and we’ll re-evaluate in a year.” This is rational caution. It’s also how you end up with deep integration by the time you realize you need an alternative. Year-one projects become standard. Dependencies calcify. By the time you re-evaluate, your systems have dependencies you forgot you had.

The Open-Source Alternative Approach: “We’ll train or fine-tune our own models.” This is intellectually satisfying and sometimes necessary. But most enterprises dramatically underestimate the cost. Training a capable foundation model requires infrastructure, talent, and capital that’s only cost-effective if you can amortize it across many applications or many customers. For a single product or department, it’s usually more expensive than staying dependent.

None of these approaches actually solves the dependency problem. They just mask it differently.

What Changes When You Design for Portability

The companies that are handling this well have made a structural decision: they’re building abstraction layers.

Instead of writing code that assumes a specific foundation model’s API response format, they write code against an abstraction that any foundation model interface could satisfy. It’s more code. It’s slightly less performant. It’s definitely more complex in the short term.

But it has one critical property: if the economics of your primary model change, or if the capabilities diverge, or if a better alternative emerges, you can actually migrate. It’s not effortless. But it’s possible without rebuilding your product.

This requires:

1. An abstraction layer at the application boundary that decouples your code from specific model APIs.

2. A cost model that includes the switching overhead in your unit economics. If you can only switch at 40% cost premium, that’s your real cost baseline, not the current API pricing.

3. A technical strategy that explicitly plans for model rotation. Not “if” but “when.” Assume you’ll need to switch models at least once over the product’s lifetime.

4. Monitoring around model performance that’s independent of vendor claims. You need to know when a model’s behavior changes, not when the vendor announces it.

This isn’t about paranoia. It’s about building businesses that can adapt when the infrastructure under them changes. Platform dependencies are real, and they’re not unique to foundation models. But foundation models are new enough that most enterprises haven’t yet built the organizational practices to manage them.

The Conversation You Should Be Having Now

The question isn’t “should we be dependent on a foundation model?” You should be. They’re better at many tasks than anything you can build or buy.

The question is: “What’s our dependency management strategy?”

That’s a conversation between your CTO (who owns the architecture), your CFO (who owns the total cost), your product head (who owns customer impact of changes), and your board (who owns risk). It’s not primarily a technical conversation. It’s a strategy conversation.

The companies that will compound value from AI over the next five years won’t be the ones that built the fastest integration in 2024. They’ll be the ones that built it in a way they can still change in 2026.

There’s a moment in every enterprise AI deployment when someone asks: “So we’ll monitor the model, right?”

And then they spend the next meeting explaining what they mean by that.

It’s one of those words that sounds simple but means completely different things depending on who’s using it. When a data engineer says they’re “monitoring” a database, they mean something specific: is it up? Is it responding? Are queries returning in acceptable time? Is storage growing unexpectedly?

When a data scientist says they’re “monitoring” a model, they usually mean: is the model’s performance degrading? Is the data distribution changing? Are we hitting the thresholds we set?

These are valuable questions. But they’re also not the questions that usually matter most in enterprise AI.

Here’s what I’ve learned: the guardrails that fail first in production are not the ones built on model monitoring. They’re the ones built on monitoring the wrong signals.

Most enterprises build their AI observability around technical metrics. Model performance, drift detection, data quality. These are important. But they’re also the ones that usually don’t surface the problems that actually matter.

Let me give you a concrete example of what I mean.

A financial services company deployed a system to recommend investment allocations. The model was good. It predicted portfolio performance accurately on historical data. In production, they monitored drift: is the model’s performance degrading? They monitored feature distributions: is the data it’s seeing different from what it trained on?

For three months, everything looked fine. No drift. Data distributions stable. The model was performing exactly as expected.

But during month four, a compliance officer pulled a sample of the system’s recommendations and noticed something. The allocations were technically sound, but they were increasingly concentrated in a narrow set of holdings. The model wasn’t breaking. But it was drifting behaviorally—making increasingly concentrated recommendations without the organization realizing it.

They only caught it because someone looked at actual recommendations. Not because the monitoring systems flagged anything.

Here’s what actually needs to be monitored in production AI systems:

First, signal integrity. Is the model receiving the data it expects to receive? Not just: is the data schema correct? But: is it coming from the same sources? Are the sources themselves changing? I know of a system that stopped working not because the model broke, but because the upstream data pipeline was silently modified and suddenly all the features were null. The model kept running. It kept producing predictions. Nobody noticed for weeks because they weren’t monitoring whether the input signals were actually present.

Second, decision consistency. When the model makes decisions, are they consistent with what you’d expect? This is qualitative. You can’t automate it. But you need someone periodically looking at actual decisions the system is making and asking: does this look right? I saw a recommendation system that was technically working fine but had learned to make increasingly extreme recommendations to a small segment of users. The metrics didn’t catch it. A human looking at recommendations caught it immediately.

Third, outcome correlation. You built a system to predict X and optimize for Y. In production, are the outcomes you’re getting consistent with the optimization? Or has the system learned a pattern that makes the metric go up but doesn’t produce the desired outcome? I’ve seen models that minimized their loss function beautifully but maximized customer churn because they learned a correlation that was mathematically true but operationally wrong.

Fourth, distribution shift that matters. Most systems monitor whether the input distribution has changed. But not all distribution shifts matter equally. A shift in the age distribution of your customer base might matter. A shift in the geographic distribution might not. You need to monitor distribution shifts relative to what you’re trying to predict, not just whether anything changed.

Fifth, the rare cases. Your model works great on the common case and terribly on the rare case. In production, are you tracking how the model behaves on the cases that are actually rare and important? Or are you only looking at average performance across everything? A lot of systems fail silently on the 2% of cases that actually matter most.

The organizations that are ahead on AI observability aren’t the ones with the most sophisticated monitoring systems. They’re the ones that are clear about what signals actually matter.

And those signals usually aren’t in your standard model monitoring dashboard.

Here’s what I’d recommend: build your monitoring in layers.

Layer 1: Infrastructure monitoring. Is the system up and running? Is it fast? This is table stakes. Use standard database and application monitoring.

Layer 2: Technical drift monitoring. Is the model’s performance degrading? Is the input data distribution changing? These are important. Use standard MLOps tools.

Layer 3: Decision monitoring. What is the system actually deciding? Are the decisions consistent with what we expect? Is the decision distribution drifting? This usually requires custom instrumentation.

Layer 4: Outcome monitoring. Are the outcomes we’re getting from the system aligned with what we wanted? This requires tracking the actual business outcomes, not just the system’s metrics. This is often the hardest layer to build, but it’s the most important.

Most enterprises have Layer 1 and Layer 2 down. They’re weak or absent on Layer 3 and 4.

That’s where the guardrails fail.

The reason is simple: Layer 1 and 2 are about the system. Layer 3 and 4 are about what the system actually does in the world. And those are different problems with different solutions.

So when someone says “we’re monitoring the model,” ask: which layer are you monitoring? If the answer is “just technical drift,” you’re missing where the real risk lives.

Build guardrails around signals that matter. Track actual decisions. Watch for outcomes that diverge from intent. Layer those on top of your standard monitoring.

The systems that fail quietly aren’t the ones where the model broke. They’re the ones where nobody was watching the right signals.

I was in a meeting with a general counsel last month. The company was deploying a large language model as an internal knowledge assistant. It was a reasonable project. The team had thought through data security. They’d built audit trails. They’d set usage policies.

The GC asked a question that stopped the room: “What happens if the model hallucinates and gives someone bad advice they act on?”

Not a weird question. A straightforward question. And it turned out nobody had a clear answer.

The conversation that followed was instructive. The data science team said: well, it’s a language model, hallucinations are a known limitation. The product team said: we have a disclaimer. The legal team said: disclaimers don’t protect us from liability.

And that’s where it usually breaks down.

Many organizations understand that AI systems can fail. What many are still developing their understanding of is the structure of liability when they do.

Here’s what I think you need before any AI system goes live in your organization:

A hallucination liability assessment.

This isn’t model documentation. This isn’t a risk register. This is a specific document that your legal team writes, not your data science team, that answers one question: if this system confidently says something that’s false, and someone acts on it, what is our legal exposure?

That’s the starting point. Because hallucination—when a language model generates confident falsehoods—is a specific kind of failure mode that most organizations haven’t built legal thinking around yet.

For decades, organizations have built legal frameworks for system failures: databases go down, processes fail, data gets lost. These are understood. There are precedents. There are insurance products.

But a system that confidently tells someone something false is a newer problem. And the legal structures around it are still settling.

Here’s why it matters:

If your system is purely informational (you’re using it to summarize documents internally), the risk profile is one thing. Someone reads the summary, notices it’s wrong, corrects it. Inconvenient. Not necessarily a legal problem.

But if your system is integrated into a workflow where a human is supposed to verify the output, but the human reasonably relies on the system’s apparent confidence, you have a different risk structure.

And if your system is trained on your proprietary knowledge and someone acts on its hallucinatory output in a way that damages a customer relationship, a business deal, or a compliance outcome, you have another risk structure entirely.

These aren’t hypothetical. I’ve watched organizations have exactly these moments. “The model told our sales team that we offered a service we don’t actually provide.” “The system generated a draft contract clause that contradicted our terms but the team didn’t catch it.” “The model synthesized customer data in a way that suggested we had capabilities we don’t have.”

None of these broke the system. All of them created liability.

So before you deploy, you need legal to answer:

First, context: What decisions or actions does this system enable or inform? What’s the chain from system output to human decision to real-world consequence?

Second, confidence: When this system is wrong, is it obviously wrong? Or does it confidently present falsehoods? (This matters because overconfident falsehoods create liability that uncertain outputs don’t.)

Third, verification: In the workflow, is there a verification step? And critically, is that step designed to catch hallucinations, or just to audit that the system ran?

Fourth, scope: Who can see the output? Is it constrained (internal team) or broad (customers, partners)? Scope multiplies liability.

Fifth, exposure: If the system hallucinates and someone acts on it, what’s the worst-case consequence? Client relationship? Compliance violation? Wrong regulatory filing? Dollar impact? This determines what you need to mitigate.

Once you’ve answered these, you’re in a position to actually assess whether you should deploy this system as-is, or whether you need to change something about how it’s used.

Maybe you need human verification at a specific step. Maybe you need to constrain who can access the output. Maybe you need specific disclaimers that are actually meaningful. Maybe you need to train people on the limitations. Maybe you need to not deploy it the way you initially planned.

Most organizations don’t do this. They get to deployment and they ask: does it work? And if it works, they deploy it. Then they’re surprised when the liability question comes up in month four because someone acted on something the system hallucinated.

Legal teams are increasingly asking for this, but they’re often asking late. The system is built. The deployment is planned. Now they’re trying to add guardrails to something that was designed without legal thinking.

That’s expensive and it’s slow. Better to do it upfront.

Here’s what I recommend: before you move a large language model (or any AI system with significant hallucination risk) from pilot to production, have your legal team write a one-page assessment. Not a binder of documentation. Not a risk matrix. One page. What’s the hallucination exposure? What are the paths from model output to liability? What mitigations do we need?

Then, once you have that clarity, the engineering team can decide: do we change the workflow? Do we add verification? Do we constrain access? Do we add specific training? Do we add disclaimers that actually mean something?

The companies that are handling this well aren’t the ones with perfect models. They’re the ones that are clear about the risk, and clear about what they’ve done to manage it.

And that clarity starts with a document that your legal team owns, not your data science team.

Get that document written before you go live. It will change how you deploy. And it might prevent you from being the organization that learns the liability lesson the hard way.

It was a Wednesday afternoon in late October when someone noticed something odd.

A risk officer was reviewing a batch of decisions made by a lending recommendation system. The system had been live for eight months. It was working. Models had been validated. Metrics looked good. But in this particular batch, something didn’t add up.

There was a pattern of approvals that shouldn’t have been approved.

The system had learned to optimize for one thing (approval velocity), and in doing so it had learned to weight certain borrower profiles in a way that made internal sense but created external risk. It wasn’t broken. It wasn’t defrauding anyone. It was just… quietly making decisions that looked right by the metrics the organization was monitoring, but created risk the organization hadn’t accounted for.

When they dug in, they found that over eight months, this system had recommended approval decisions on credits that had characteristics which, under scrutiny, suggested higher default probability than historical norms. The organization had essentially been taking on tail risk that nobody had measured because nobody was measuring the right thing.

The cost, when they finally calculated it, was in the millions. Not because the model was wrong. But because the operational reality of how the system was being used diverged from how it was supposed to be used.

This is not a story about a specific bank. But it’s a pattern I’ve seen map across different domains, different risk types, different sizes of organizations. And it illustrates something fundamental that I think enterprise leaders chronically underestimate: operational risk in AI systems is different from model risk.

Model risk is the question: does the model predict accurately? You can measure that. You can validate it. You can test it on hold-out data.

Operational risk is the question: what actually happens when humans and machines make decisions together in the real world, over time, at scale?

It’s much harder to measure. And it’s where the real money goes.

Here’s how it usually happens:

The system is deployed. It works. Engineers and data scientists are satisfied because the metrics are holding. The business is satisfied because the system is processing decisions faster. Everyone feels good.

Then, slowly, a gap opens up between how the system is being used and how it was designed to be used.

Maybe a human decision-maker starts trusting it too much and stops applying their judgment. Maybe they start using it in cases it wasn’t designed for. Maybe the process changes and the system isn’t updated. Maybe the data it’s learning from drifts because the business has shifted and nobody told the model.

These aren’t failures. They’re just reality. Real humans, real workflows, real organizations. Things change. People adapt. Systems… sometimes don’t.

And in the gap between how the system was designed and how it’s actually being used, risk accumulates. Silently.

The lending scenario is clean because the risk is quantifiable. But the pattern is broader.

A claims adjudication system that learned to approve claims faster starts approving claims that shouldn’t be approved. A hiring system that optimizes for filling positions quickly learns to screen out categories of applicants that were never the intent. A fraud detection system that minimizes false positives starts letting real fraud through. A customer retention system that minimizes churn learns to make offers that are economically senseless.

None of these happen because the model broke. They happen because the operational reality diverged from the model’s intent.

So how do you prevent it?

The honest answer is: you can’t, completely. Humans are adaptive. Systems are rigid. Gaps will open.

But you can reduce it dramatically by doing something most organizations don’t do: operationalize decision monitoring.

Not model monitoring. Decision monitoring.

Model monitoring is: is the model drifting? Is the data quality declining? These are good questions. Most enterprises do some version of this.

Decision monitoring is different: what decisions is the system actually making? Are they consistent with what we expected? When the system and humans disagree, what’s the pattern? What are we learning about how this system is actually being used?

This requires infrastructure that most organizations don’t have. You need to log not just the model’s prediction, but the human’s decision. You need to understand the gap between them. You need to understand how that gap is changing over time.

Then you need someone—an actual human with authority—reviewing this regularly. Not algorithmically. Manually. Reading through cases. Asking: does this feel right? Where should I worry?

The lending organization I referenced earlier, once they figured out what had happened, implemented this. They assigned a risk officer to review 50 random decisions per week from the system. Just reading them. Just asking: do these approvals make sense? Do they align with our risk appetite?

That single investment—one person’s time—would have caught the drift before it became millions in tail risk.

Most organizations won’t do this. It feels expensive. It feels like overhead. The system is working. The metrics are good. Why do you need a human reading cases?

Because operational risk isn’t in the metrics. It’s in the gap between what you’re measuring and what’s actually happening.

Here’s the hard part: you probably can’t see that gap from the metrics. You have to see it by looking at actual decisions, actual outcomes, actual patterns in how the system is behaving in the world.

So if you deploy an AI system that makes real decisions—lending, hiring, fraud, claims, pricing—you need someone whose job is to catch the operational risk. Not to prevent the model from drifting. To catch the moment when the organization’s use of the system starts diverging from the system’s design.

It’s boring work. It won’t show up in quarterly metrics. It’s also how you avoid being the organization that discovers, in month eight, that you’ve been taking risks you didn’t know about.

The cost of operational risk isn’t usually in one bad decision. It’s in the pattern of decisions that are individually defensible but collectively add up to exposure you didn’t want.

Pay attention to that gap. Assign someone to watch it. It’s cheaper than learning the lesson the hard way.

Every data science team has a graveyard.

You won’t see it in the org chart or the budget forecasts. It’s the projects that worked—on paper. They hit their targets. The models converged. The metrics looked clean. Then they got four months into production and somehow, without anyone quite understanding when it happened, they became optional.

A recommendation engine that no one acts on anymore. A churn prediction model that technically has good precision but the business stops using it because it “doesn’t feel right.” A customer segmentation model that’s accurate by every measure except the one that actually matters: whether the business can do anything useful with the segments.

If you ask the data science team what went wrong, they’ll tell you the truth: nothing went wrong. The metrics didn’t move. The model didn’t degrade. Everything performed as expected.

And that’s the problem.

The wrong metrics don’t fail. They persist. They sit there, humming along, proving they work, while the business quietly stops trusting them.

Most enterprise teams measure AI success through a lens designed for a different problem: Can we build a model that predicts X with accuracy Y?

That’s a data science problem. It’s not an enterprise problem.

An enterprise problem looks like: Can we deploy a system that helps a human make better decisions, at a cost we’re willing to pay, in a way that creates actual value?

These are not the same question. And the gap between them is where value goes to die.

Here’s what happens. Your team builds a credit risk model. They measure it on hold-out test data. Precision, recall, AUC—all of it beautiful. 0.92 AUC. You can draw the ROC curve and it’s smooth and clean.

Then it goes live. And in month two, a credit analyst pulls you aside and says: “The model is right more often than I am. But I’m the one who has to justify the reject to the customer. When the model says no and I say no, I sound like a robot reading numbers. But I can explain why in a way they trust.”

And suddenly that 0.92 AUC doesn’t matter. What matters is whether the credit analyst will use the model or work around it.

Most organizations don’t measure that. That’s not a metric. That’s an adoption problem. And adoption problems are usually blamed on the humans, not the model.

“If they’d just trust the model more,” you hear. “If they understood how accurate it is.”

But the humans understand the metrics perfectly. They’re just answering a different question: does this reduce my risk while I do my job?

The real cost of measuring success the wrong way shows up later. You’ve deployed 47 models. By your metrics, 44 of them work. By the organization’s metrics—are they actually being used in production decisions, are they creating value, do people trust them—maybe 12 work.

You’ve spent millions on infrastructure, data science teams, model governance frameworks. You’ve built a technically sophisticated machine that delivers technically perfect predictions nobody cares about.

This is why so many enterprise AI programs feel expensive and slow and produce results that never quite justify the investment.

It’s not that the organizations are incompetent. It’s that they’re measuring success at the wrong level.

You need at least three measurement frameworks running in parallel:

The first is technical. Does the model predict accurately on hold-out data? Can it handle the data it receives in production? Does it drift? These are table stakes. But they’re not sufficient.

The second is behavioral. When the model makes a recommendation or a decision, what does the human do? Do they follow it? Do they override it? Do they use it as input to their own judgment, or do they ignore it? How often do they come back and say “you were right” vs. “that was wrong”? If your model is predicting well but humans override it most of the time, you don’t have a model problem. You have a deployment problem. But you only see it if you’re measuring it.

The third is economic. Does using this model reduce costs, increase revenue, or improve customer outcomes by an amount greater than what we spent building it? You’d be shocked how many enterprise teams don’t know the answer to this question. They have a model. It works. But has it actually paid for itself?

The cost of getting this wrong is two-fold. First, you waste money on systems that don’t deliver value. Second, you damage the credibility of the whole enterprise AI program. You build a reputation for expensive, sophisticated systems that don’t work. And “work” in the minds of decision-makers means: does it help me do my job better?

The organizations measuring AI success effectively aren’t the ones with the best data scientists. They’re the ones that are clear about what “success” means before they start. They’re clear about it in a way that’s measurable, observable, and tied to something the business cares about.

A head of AI I know runs a quarterly check on every model in production. It’s simple. She asks: Is anyone using this? Are they using it the way we expected? Is it creating value? If any of those answers is no, they either fix it or they retire it.

No heroes. No “it’s technically correct so we must be doing something wrong.” Just: does this work in practice?

That discipline—measuring success in three dimensions, and being willing to retire systems that work on metrics but not in reality—is what separates programs that produce value from programs that produce impressive dashboards.

Your next AI project? Decide what success looks like before you start. Make sure at least one dimension of it is something humans have to care about. Then measure it, all the way through.

The metrics will take care of themselves.

There’s a moment in every enterprise AI deployment when someone walks into a meeting with their arms crossed and says: “Our data quality isn’t good enough.”

What they mean is: “I’ve looked at the source systems. Some fields are empty. Some are wrong. Some mean different things depending on which department entered them.”

And then everyone nods solemnly, as if this is a surprising discovery. As if 1980 called and wants its data problems back.

The lie isn’t that you have bad data. Of course you do. The lie is the myth of good enough.

Most organizations operate under a belief that’s almost never stated explicitly, but it sits there like furniture in the room: if you can get your data to 90% quality, you can build a functional AI system. Maybe 85%. Definitely at 95%.

This is false.

It’s not false because data quality doesn’t matter. It’s false because you can build a functional AI system with pretty broken data—you just can’t build one you can trust to make decisions at scale.

I’ve watched teams spend eight months fixing data pipelines. Getting the nulls down. Standardizing the schemas. Running reconciliation checks. And when they get the system into production, it performs fine on metrics. It produces outputs. But when a human looks at what it’s actually recommending—especially in edge cases, in the long tail—there’s something off about it. Not catastrophically, not always. Just… off.

The system learned the pattern, including the pattern of your data quality problem.

Here’s what’s actually happening underneath. Your data has three types of defects:

Structural defects are easy. Missing values, wrong data types, things that break schemas. Your tools catch these. You fix them. Life goes on.

Semantic defects are harder. A field labeled “revenue” that means net revenue in one business unit and gross in another. A “customer type” that was coded one way three years ago and a different way now, but nobody updated the historical data. A date field that’s sometimes a transaction date and sometimes a creation date, depending on which system it came from. These are invisible to structural checks. They look clean on a data quality scorecard. And they teach your model nonsense.

Temporal defects are the ones that kill you. Your historical data is, by definition, old. It was collected under different business conditions, different processes, different market dynamics. An AI system that learns from it is learning from a past that no longer exists. You’ll discover this about six weeks after your system goes live, when its predictions start diverging from reality.

The real question isn’t: is our data good enough?

It’s: what decisions are we letting this system make, and what defects can we actually live with?

This is where most conversations break down. Because it forces you to be specific. Specific means uncomfortable.

A recommendation engine that occasionally suggests a slightly irrelevant product? You can live with that. Your customers see a mediocre recommendation, they ignore it. Cost of that failure: low.

A system that scores credit risk or flagged fraud, and it’s learned to be structurally inconsistent about what “fraud” means? You can’t live with that. You’ll catch it in compliance review three months in and have to rebuild it.

The practical move isn’t to announce you need “better data quality.” That’s governance theatre.

The practical move is to map your data defects against the specific decisions you’re making. Ask yourself: where does this data come from? Who was incentivized to enter it, and what were they incentivized to do? What changed three years ago in the system that collected this? Where is it inconsistent, and what does that inconsistency mean for what we’re trying to predict?

Then: deliberately reduce the scope of decisions you let the system make until the defects stop mattering.

You might keep the system from making binary decisions and make it advisory instead. You might restrict it to certain business units until you understand why the data behaves differently there. You might run it in shadow mode longer than you planned because the temporal defects are real and you want to see how it drifts.

This is boring. It’s not a feature announcement. It won’t make the quarterly exec summary.

But it’s the difference between a system that hums along in production for three years and one that breaks down in month eight when someone realizes it’s been quietly making recommendations that contradict its own logic because your semantic defects taught it two contradictory patterns.

Your data quality problem isn’t that you need better data. It’s that you need specific knowledge about where bad data actually matters, and you need the organization discipline to say: not yet. Not there. Not until we understand this.

That’s harder than buying a data quality tool. And it’s exactly the kind of hard that distinguishes deployments that compound value over time from the ones that become cautionary tales.

This isn’t a joke. It’s how many AI deployment decisions actually happen in enterprises, and it ends the same way: with nobody being entirely clear about who made the call.

The CFO’s concern is straightforward: if this fails, how much does it cost us? They want to understand the financial consequences. They’re thinking about downside risk, capital allocation, shareholder communication if something goes wrong.

The VP of AI is thinking about capability and execution. Can we actually build this? Will it work? What will we learn? They’re thinking about staying competitive, building the team’s skills, proving value. They want to move.

The Compliance Officer is thinking about exposure. What if a regulator asks about this? What if a customer complains? Can we defend this decision? They’re thinking about policies, documentation, the audit trail. They want to be careful.

These aren’t contradictory perspectives. But they’re different perspectives, and they weight risks differently. The CFO is most concerned about economic risk. The VP is most concerned about execution risk. The Compliance Officer is most concerned about regulatory risk. When you put them in a room to decide whether to deploy an AI system, someone has to make the call. And often, nobody actually does.

What usually happens instead is consensus seeking that masks disagreement. You leave the meeting with implicit agreement that “we’re moving forward,” but the CFO is thinking “we’re moving forward if the economic model works out,” the VP is thinking “we’re moving forward and we’ll figure out the details,” and the Compliance Officer is thinking “we’re moving forward and we’ll handle the risks.” These statements sound like agreement. They’re not. They’re three different decisions that will conflict the moment execution requires actual tradeoffs.

Then something happens—the model drifts, a customer complains, a regulator asks a question—and suddenly it’s clear that you never actually resolved the disagreement. You just deferred it.

The Three Types of Risk These People Actually Represent

To understand why this happens, start with what each person is actually accountable for.

The CFO is accountable for financial performance. If the AI system costs more than it saves, they answer to the board. If it creates unexpected costs, they’re explaining variance to investors. Economic risk is their domain.

The VP of AI is accountable for delivering on the AI roadmap. Building capabilities, shipping systems, enabling the business. Execution risk is their domain. They’re being evaluated on delivery and impact, not on managing risk.

The Compliance Officer is accountable for regulatory compliance and risk management. If something goes wrong and there’s no documented governance, they’re explaining to auditors and regulators. Governance and procedural risk is their domain.

These aren’t competing priorities. They’re different dimensions of risk. The problem is that your governance structure probably treats them as competing. It probably puts them in the same meeting, assumes they’ll reach consensus, and then acts surprised when they don’t.

The real issue is that nobody is explicitly accountable for integrating these perspectives and making the actual decision. The CFO can’t decide alone because they don’t understand execution risk. The VP can’t decide alone because they’re not responsible for financial outcomes. The Compliance Officer can’t decide alone because they’re not responsible for business outcomes. So you have three people none of whom is empowered to make the call.

This is a different governance problem than the ones in previous pieces. In Days 1-2, I was focused on the structural problem: nobody owns the decision. Here we’re looking at the substantive problem: when you have multiple valid perspectives and they don’t align, how do you actually decide?

What Happens When You Pretend Consensus Exists

Most organizations try to solve this through consensus. The theory is: if everyone agrees it’s a good decision, it’s a good decision. If people disagree, talk until they agree.

This works for some decisions. It doesn’t work for decisions where people have legitimately different risk appetites.

Say the AI system will cost $500K to build and maintain, and it will save $1.5M annually if it performs as modeled. The CFO looks at this and says: “If performance is as modeled, great. If it underperforms by 20%, we still have positive ROI, so I can support this.” That’s a CFO’s way of agreeing.

The VP of AI looks at the same numbers and says: “I can build this, and I believe the model will work well.” That’s a VP’s way of committing to execution.

The Compliance Officer looks at the same numbers and says: “I need to see the governance around this, but the financial case looks sound.” That’s a Compliance Officer’s way of saying they’ll proceed carefully.

All three have “agreed.” But the CFO agreed subject to financial performance. The VP agreed subject to being able to execute. The Compliance Officer agreed subject to governance being in place. If any of those conditions isn’t met, the agreement falls apart. But nobody documented the conditions. So everyone walks out of the meeting thinking they’ve committed to moving forward, when what they’ve actually committed to is moving forward if certain things happen.

Then execution reveals that governance isn’t as clear as the Compliance Officer wanted, or execution is harder than the VP expected, or the financial model is too optimistic. Now you have a decision you thought was made, coming apart under the stress of reality.

The Actual Decision-Making Process That Works

The organizations I’ve seen navigate this successfully have a structure that acknowledges this complexity rather than hiding from it.

They explicitly assign decision authority to one person. Often this is a business leader—a VP of Product, a business unit leader, a Chief Operating Officer. Someone who’s accountable for outcomes, not just for process. That person is explicitly accountable for the deployment decision.

They structure the input so that person gets clear perspective from all angles. Not consensus—perspective. The CFO provides financial analysis and financial risk tolerance. “Here’s what happens if the model underperforms by 10%, 20%, 30%. Here’s the financial exposure.” The VP of AI provides execution assessment. “Here’s our confidence in our ability to build this. Here’s what could go wrong technically. Here’s how we’d mitigate it.” The Compliance Officer provides governance and regulatory assessment. “Here’s what we need to document. Here’s what regulators might ask. Here’s what we can defend.”

The decision-maker’s job is to integrate these inputs, understand the tradeoffs, and make a call. Not “do we all agree?” but “given what we know, should we move forward?”

They document the decision explicitly. Not “we decided to deploy,” but “we decided to deploy this system, accepting these financial risks, with this mitigation plan, having documented this governance, knowing that these scenarios could happen and here’s how we’ll handle them.” That decision document becomes the reference point when things change. If something goes wrong, you can point to what you understood and accepted at the time.

They assign someone to own ongoing accountability. Usually the same person who made the deployment decision. That person is accountable not just for the decision being made, but for it being monitored and for adjustments when things change. They’re the person who can actually pull the system if something goes wrong.

The Specific Governance Moments Where This Breaks

In my experience, this breaks down in specific places:

The decision meeting that doesn’t resolve anything. You’re in a room. Three perspectives. No explicit decision authority. You talk until everyone is tired, and then you say “okay, sounds like we’re aligned on moving forward.” You’re not aligned. You’re just tired. Write down what you’ve agreed to. Document the conditions and assumptions. Have the decision-maker say explicitly: “Here’s what I’m deciding, and here’s what I’m accepting by making that decision.”

The governance framework that exists but doesn’t actually gate anything. You have a checklist. Model validation: check. Fairness assessment: check. Compliance review: check. And now you’re automatically approved to deploy. This is governance theater. Someone has to actually make a decision based on the checklist. Not “has the checklist been completed,” but “given what the checklist shows, should we deploy?” If the answer is always “yes,” you don’t have governance. You have a process.

The risk discussion that’s all upside and no downside. You’re talking about how much value this will create. Nobody’s talking about what happens if it doesn’t. The CFO should be laying out financial downside. The VP should be laying out execution risk. The Compliance Officer should be laying out governance and regulatory risk. If everyone is only talking about upside, you haven’t actually thought about risk. You’ve just decided you like the idea.

The monitoring plan that nobody actually monitors. You’re going to “track accuracy” and “monitor for bias” and you’ve assigned it to a team. That team has other priorities. Months later, nobody’s looking at the metrics. And the first time something goes wrong is the first time anyone actually paid attention. Assign actual, prioritized monitoring to someone. Not “the data science team will keep an eye on it,” but “person X is responsible for checking this metric weekly, and person Y is responsible for deciding what to do if the metric moves.”

The escalation that doesn’t actually escalate. Something unexpected happens with the model. The monitoring team sees it. They send an email. It gets lost. Or it goes to a stakeholder who doesn’t have the authority to make decisions. Or it gets escalated to a committee that meets quarterly. By then it’s late. Build an escalation path where an observation actually reaches someone with authority to act, and with enough urgency to matter.

How This Looks in Practice

Here’s a real-world example of what I mean (composite, not a specific case):

You’re deploying an AI system for credit decisions. The CFO looks at the model and says: “If this achieves 85% approval rate with a default rate of 4%, I can fund this. If default rate goes above 5%, we need to recalibrate or stop.” That’s the financial decision rule.

The VP of Risk (who owns credit) says: “I can defend these decisions if I can explain them. If we’re using a black-box model and regulators ask why we denied someone credit, I need an explanation. I need model documentation and decision-level traceability.” That’s the governance decision rule.

The Compliance Officer says: “Fair Lending is a regulatory focus. We need to demonstrate that our approval rate is not substantially different across demographic groups, or if it is different, we need to explain why based on credit risk factors, not demographic factors.” That’s the regulatory decision rule.

The decision-maker’s job is to say: “We’re deploying this system. The target is 85% approval with 4% default. We’re committing to documenting decisions and having explainability. We’re targeting demographic parity adjusted for credit risk. If either of the first two gets violated, we recalibrate. If the third gets violated, we stop. Person A monitors the approval rate and default rate weekly; person B monitors the decision documentation and explainability; person C monitors the demographic parity. If any metric moves outside acceptable range, person A/B/C escalates to me for decision.”

That’s a decision. Not a consensus that hides disagreement. A decision that integrates different perspectives and lays out what happens next.

The Board Question

When the board asks “why did you deploy that AI system?” the answer should be specific:

“We evaluated the deployment against financial risk, execution risk, and governance risk. The financial model showed positive ROI with acceptable downside. We had sufficient technical capability to execute. We had governance and regulatory framework in place to defend the decision. We assigned clear ownership and escalation. We made the decision deliberately, documenting what we understood and accepted.”

That’s what a governance decision sounds like. Not “everyone agreed,” not “the model was validated,” not “it was on the roadmap.” But “we thought through the decision, we integrated perspectives from across the organization, and we were willing to own the consequences.”

When you have that clarity—about who decides, what they’re deciding, what they’re accepting, what happens if things change—the CFO, the VP, and the Compliance Officer stop needing to reach consensus on something they never fully agreed about in the first place. They each provide clear perspective on their domain, someone integrated that perspective into a decision, and they all know what was decided and why.

That’s how you actually handle multi-stakeholder AI deployment decisions. Not perfectly. But deliberately.

Most organizations answer this question with metrics. Accuracy. F1 score. AUC. Precision and recall. So they build validation processes optimized for those numbers. They backtest them. They test on holdout datasets. They measure them on new data. And then they watch the metrics carefully after deployment.

Here’s the problem: you’re measuring whether the model works. You should be measuring whether the decision it produces is acceptable.

These are different things.

A model can be technically accurate and still produce decisions that violate your risk tolerance. You can have high F1 scores and low decision quality. You can reduce bias metrics to perfect parity and still fail because you’re optimizing for the wrong fairness measure, or you’re achieving fairness in a way that destroys business value, or you’re failing on a different fairness measure you didn’t think to check.

The organizations that actually manage model risk well aren’t the ones with the most sophisticated validation metrics. They’re the ones that ask a different question first: what decision quality do we require? And then: how do we know if we’re achieving it?

The Accuracy Trap

Start with accuracy, because it’s the easiest one to get wrong.

You’ve trained a model to predict something. It’s 95% accurate on test data, 94% on a new holdout set, 93% after the first month in production. These are good numbers. And they’re technically meaningless for understanding your risk.

What matters is what happens when the model is wrong. If the model is predicting customer churn, and it’s 93% accurate, that means it’s wrong 7% of the time. That 7% breaks into two categories: people it predicted would churn who didn’t (false positives), and people it predicted wouldn’t churn who did (false negatives). These aren’t symmetric costs.

A false positive means you treat a loyal customer as a churn risk. You might offer them discounts, special treatment, outreach. That’s usually not expensive. It’s annoying marketing waste.

A false negative means you miss a customer who’s actually going to churn. You don’t intervene. They leave. That’s actual business loss.

So from a business perspective, 93% accuracy in a churn model might be a disaster (if the 7% errors are mostly false negatives) or acceptable (if they’re mostly false positives).

Your validation metrics don’t tell you this. You measure accuracy, and it’s high. You’ve validated the model successfully. And then in production, you’re failing at the business problem you were trying to solve, because you optimized for the wrong metric.

This scales up. In lending, false negatives (denying credit to someone who would have repaid) cost you revenue. False positives (approving credit for someone who defaults) cost you loss. The acceptable balance depends on your risk appetite, your business model, your regulatory environment, your customer expectations. Not on what the model’s F1 score is.

In hiring, false positives (rejecting candidates you would have wanted) destroy your talent pipeline and waste recruiting costs. False negatives (hiring people who don’t succeed) destroy your teams. The acceptable ratio depends on your hiring philosophy, your training pipeline, your attrition rates, your regulatory obligations. Again: not on the accuracy metric.

Most validation processes don’t actually think about this. They measure accuracy as a proxy for quality, and they assume that if the metric is good, the decision is good. It often isn’t.

The Fairness Metric Confusion

This gets worse with fairness, because fairness metrics sound more sophisticated but are actually more fragile.

You’ve committed to making fair decisions. Great. Now: what does fairness mean?

Equal accuracy? The model predicts defaults equally accurately for applicants of all races, genders, ages. Sounds good.

Except: if default rates are actually different across groups (maybe due to historical discrimination, maybe due to real difference in risk, depends on the domain), then equal accuracy means equal Type I and Type II error rates, which means unequal impact. You approve proportionally more loans to the group with lower baseline default rate.

Equal impact? The model rejects the same percentage of applicants from each demographic group. Sounds fair.

Except: if rejection rates should actually be different (if the groups have different risk profiles), then equal impact means you’re being statistically inaccurate, and you’re probably violating fairness in some other direction. You’re approving loans you shouldn’t approve, or rejecting people you should approve.

Equal opportunity? The model is equally likely to identify a non-defaulter as non-defaulter, regardless of demographic group. This is sometimes called “equalized true positive rate.”

Except: this assumes that not defaulting is equally distributed across groups, which it might not be. And it doesn’t address false positive rates. And it’s a different fairness metric than the last two.

There are eight major fairness frameworks, and they’re mutually incompatible. You can optimize for one and fail on all the others. And there’s no mathematical way to optimize for all of them simultaneously.

Most validation processes pick one metric, measure it carefully, optimize for it, and call that “ensuring fairness.” Then they’re surprised when their model is challenged on a different fairness dimension. Or when their fairness metric is high but the decisions are still systematically wrong for a particular group.

The problem is the same as with accuracy: you’re measuring a proxy, and assuming the proxy is the outcome you care about. It usually isn’t.

What Actually Matters: Decision Quality

Here’s what validation should actually be checking: does this model produce decisions I’m willing to make and defend?

That’s a different question. It requires you to think through what you’re actually trying to accomplish with this model, what the constraints are, and what outcomes you need to be able to defend.

For a churn model: can you identify which customers are most likely to leave, well enough that your intervention strategy is cost-effective? Can you explain why you’re intervening on customer X but not customer Y? Can you explain what happens if you’re wrong? Can you accept the distribution of errors (false positives and false negatives) that you’re creating?

For a lending model: can you identify creditworthy applicants reliably enough that your default rate is acceptable? Can you defend the decisions you’re making about particular applicants? If the model says “reject” but you override it and approve them, how often do they succeed? If the model says “approve” but you reject them anyway, how often would they have succeeded? Can you accept the financial consequences of the error rate?

For a hiring model: can you identify candidates who will succeed in the role? Can you explain why you screened out a particular candidate? If the model says someone won’t succeed but they do (in your competitors’ organizations), how do you reconcile that? Can you accept the representation outcomes you’re creating?

These questions require business judgment and domain knowledge, not metrics. You can’t reduce them to a single number. But they’re the actual validation question.

Building the Right Validation Process

If you want to actually validate whether a model is acceptable to deploy, you need a different process:

First: Define decision quality requirements. Not metrics—requirements. What outcomes do we need this model to produce? What are we willing to accept? What are we not willing to accept? For a lending model: “We need to identify 80% of creditworthy applicants without exceeding a 3% default rate.” For a churn model: “We need to identify 60% of likely churners; we’re willing to spend on interventions that have a 50% success rate.” For a hiring model: “We need candidates with >70% 2-year success rate; we want at least 30% representation of underrepresented groups in our hire cohort.” These are business requirements, not technical metrics.

Second: Measure whether you’re meeting them. This is where metrics come in, but they’re metrics aligned with your actual requirements. Not “what’s our F1 score,” but “of the people we identified as likely to churn, did 50% actually churn?” Not “what’s our fairness metric,” but “do we have the representation we said we wanted? And at what cost to accuracy?” You’re validating that you’re meeting the specific requirements you set, not optimizing for proxies.

Third: Understand the tradeoffs you’re making. Every model involves tradeoffs. Better accuracy at the cost of fairness. Higher profit at the cost of customer risk. Faster decisions at the cost of accuracy. Most organizations don’t deliberately articulate these tradeoffs. They just happen. Build validation that makes them explicit: “If we deploy this model, we gain X in business value and we accept Y in risk. Here’s what Y looks like in concrete terms.” Make sure the person approving the deployment actually understands what they’re approving.

Fourth: Establish the failure threshold—before deployment. What would cause you to pull this model? Not “we’ll know it when we see it,” but “if [metric] exceeds [threshold], we reconfigure the model or take it offline.” This isn’t about identifying all problems; it’s about identifying the problems that matter most. You might not care if your churn model’s accuracy drifts from 93% to 90%. You probably do care if it starts making decisions that violate your fairness requirements. You definitely care if the cost per intervention increases beyond what makes the program economically viable.

What Happens When Validation Aligns with Decision-Making

Organizations that actually validate well have one thing in common: they’ve spent time thinking through what decision quality looks like for their use case, before they build the model. The validation process then checks whether they’re achieving it.

This sounds obvious. It’s not what most organizations do.

Most build the model first, validate against metrics, and then try to justify deployment based on the metrics. They work backward from the numbers they have to an acceptable story about what those numbers mean. Sometimes the story holds up. Often it doesn’t.

When someone says “our validation shows this model is ready,” what they usually mean is “we have metrics that look good.” They usually don’t mean “we have confirmed this model produces decision quality we’re willing to defend.” These are different things.

The reason this matters for governance and risk is straightforward: when something goes wrong with the model, someone will ask you to justify the deployment decision. “This model is making this decision. Why did you think that was acceptable?” Your answer can’t be “our F1 score was 0.87.” Your answer has to be “we required [X outcome], we validated that we achieved it, we understood the consequences, and we decided it was acceptable.” That’s an answer you can defend. The other one isn’t.

Your validation process isn’t just a technical gate. It’s the evidence for your governance decision. Make sure it’s measuring what actually matters.