A bank’s Chief Risk Officer pulled out a model risk framework. It was comprehensive: data quality checks, concept drift monitoring, backtesting protocols, confidence intervals, stress testing. It looked good. Very academic.

Then I asked: “What happens when a model decides to drop a product line that’s currently profitable but the model thinks is risky?”

Silence.

“What if the model’s risk assessment is more conservative than the business risk tolerance, and nobody knows that’s the conflict?”

More silence.

“Who resolves that conflict?”

This is the gap that almost every enterprise is walking into: they’re using risk frameworks designed for research or trading (where “maximize accuracy” is the goal) and applying them to production systems in regulated industries (where “make the right business decision” is the goal, and accuracy is just one input).

These are not the same problem.

Why Academic Frameworks Don’t Translate

Let me be clear about what academic model risk frameworks do well:

They identify technical failure modes. They catch when training data has changed. They detect when model assumptions are violated. They flag when a model’s confidence is misplaced. These are real, important things.

What they don’t do is address business risk. They can’t, because business risk is not the same as model risk.

In a research setting, a model is either right or wrong. In an enterprise setting, a model is making a decision that has business consequences. The risk isn’t “the model is inaccurate.” The risk is “the model is making a decision that conflicts with our business goals or regulatory constraints, and we didn’t catch it.”

Here’s the concrete difference:

A research model that fails: “The accuracy dropped from 94% to 91%. Something is wrong with the model.”

An enterprise model that fails: “The model is rejecting 15% more credit applications than it used to. Is that because credit quality changed, or because we changed the training data, or because the model is malfunctioning, or because the model’s risk tolerance diverged from the business risk tolerance?”

The research framework will catch some of this (data changes, concept drift). It won’t catch the one that matters most: we didn’t check if the model’s decision-making is aligned with what the business actually wants.

The Misalignment Nobody Talks About

Here’s what I see in most enterprise risk frameworks:

“Monitor model performance. Flag degradation. Retrain. Validate. Deploy.”

What’s missing: “Validate that the model’s decisions are aligned with business goals.”

And the reason it’s missing is that people don’t think of it as a model problem. They think of it as solved by governance. “We have a governance committee that approves what the model does.”

But the committee approves the model. They don’t continuously validate that the model is doing what it approved.

So you get scenarios like this:

A model for employee promotion recommendations gets deployed. The governance committee approved it. The accuracy is good. The fairness testing passed. It’s in production.

Six months later, someone notices the model is recommending promotions for technical specialists into management roles at much higher rates than the business actually wants. The model learned that technical expertise is predictive of success, which is true statistically. But the business doesn’t want to promote all the good engineers into management. That’s not what the business risk tolerance is.

Is this a model risk problem? In the academic sense, no. The model is doing exactly what it was trained to do. In the business sense, yes. The model’s decisions conflict with what the business actually wants.

The academic framework checks: “Is the model accurate?” Yes. “Is the model fair?” Yes. “Is the model stable?” Yes. “Is the model doing what the business wants?” Never asked.

The Risk Framework You Actually Need

Real enterprise model risk has three layers:

Layer 1: Technical Risk. Is the model working as designed? This is what academic frameworks address. Data quality, concept drift, distribution shift, confidence calibration. These matter. Monitoring these is necessary but not sufficient.

Layer 2: Business Risk. Is the model’s decision-making aligned with business goals? This is where most frameworks stop. You need to actually validate that the model is making the business decisions you want it to make, not just making accurate predictions.

Layer 3: Governance Risk. Is someone explicitly accountable for what the model does? And if the model’s behavior conflicts with policy or regulation, do you have a way to know and a way to respond?

Here’s how you’d actually monitor a credit model:

Technical layer: Monitor data quality (are the features that go into the model staying the same?). Monitor performance (is accuracy stable?). Monitor distribution (are the inputs to the model changing in unexpected ways?).

Business layer: Monitor outcomes in context. If approval rates are changing, is that because credit quality changed, or because the model changed? If the model is approving more customers from some segments than others, is that intentional? What’s the approval rate spread you’re willing to accept?

Governance layer: Is there someone who’s reviewed the outcomes and said “yes, this is what we want”? If the model is declining business that was profitable, who made that decision? If the model’s risk tolerance is more conservative than the business wants, who’s changing which one?

Why This Actually Matters for Your Business

The reason this distinction is critical is that most enterprise models are making proxy decisions.

You don’t actually want a credit model that’s “accurate at predicting default.” You want a credit model that makes profitable lending decisions, recognizes systemic risk, and complies with regulation. Accuracy is a means to those ends, not the goal itself.

If your model is 95% accurate at predicting default but it’s also declining 30% of applications that would have been profitable, you have a problem. The academic framework will say “the model is accurate.” The business framework will say “the model is costing us money.”

In a lot of enterprises right now, this is exactly what’s happening. Models that were built to be accurate are getting used to make business decisions, and nobody’s checking whether the business decisions are good.

The technical team builds a model that predicts well on the training data. It passes validation. It gets deployed. The business starts using it. Six months later, someone notices it’s making different decisions than the human process it replaced, and nobody knows if that’s good or bad.

What Changes When You Separate These Risks

The first thing that changes is accountability. If a model is making bad business decisions, it’s not “a model accuracy problem.” It’s “a governance problem” or “a business decision problem.” And the person who’s accountable is not the data scientist. It’s the person who’s using the model to make decisions.

The second thing that changes is what you monitor. You don’t just look at technical metrics. You look at decision outcomes. You build feedback loops that tell you whether the model is doing what the business wants it to do.

The third thing that changes is escalation. If the model is making technically correct predictions but bad business decisions, you need a clear path to change the model, change the policy, or change the usage. And you need someone accountable for that decision.

The Conversation to Have Now

Most enterprises have started monitoring technical model risk. That’s good. But they haven’t started monitoring business model risk, and they definitely haven’t integrated that into governance.

The conversation you should be having is not: “Do we have good model monitoring?” It’s: “Do we know whether our models are making the business decisions we want them to make?”

That requires:

1. Clear business objectives for each model. Not “predict well.” What decision do we want the model to make? What constraints does it have? What risks are we accepting?

2. Outcome monitoring that’s connected to business metrics. If the model changes its behavior, do we know? If the decision outcomes change, is that intentional?

3. Escalation clarity. If the model’s decisions diverge from the business goals, who has authority to change the model or change the policy?

4. Explicit ownership. Not “the data science team owns the model.” Someone owns “we’re using this model to make X decision and it’s working as intended.”

This is not about building more sophisticated risk frameworks. It’s about connecting the technical risk work to the business decision that’s actually being made.

The Risk That Looks Like Success

The most dangerous scenario is the one where the model is technically sound, it’s making consistent decisions, it passes all the monitoring checks—and it’s making decisions the business doesn’t actually want.

You won’t notice this until it matters. Until someone asks, “Why did we decline that credit application?” and the answer is “the model said so,” and the question back is “but is that what we actually want?”

If you can’t answer that quickly and clearly, you have a model risk problem that no amount of academic monitoring will catch.