In Practice: AI in the Enterprise | Day 6: The Data Quality Assumption We All Make

There’s a moment in every enterprise AI deployment when someone walks into a meeting with their arms crossed and says: “Our data quality isn’t good enough.”

What they mean is: “I’ve looked at the source systems. Some fields are empty. Some are wrong. Some mean different things depending on which department entered them.”

And then everyone nods solemnly, as if this is a surprising discovery. As if 1980 called and wants its data problems back.

The lie isn’t that you have bad data. Of course you do. The lie is the myth of good enough.

Most organizations operate under a belief that’s almost never stated explicitly, but it sits there like furniture in the room: if you can get your data to 90% quality, you can build a functional AI system. Maybe 85%. Definitely at 95%.

This is false.

It’s not false because data quality doesn’t matter. It’s false because you can build a functional AI system with pretty broken data—you just can’t build one you can trust to make decisions at scale.

I’ve watched teams spend eight months fixing data pipelines. Getting the nulls down. Standardizing the schemas. Running reconciliation checks. And when they get the system into production, it performs fine on metrics. It produces outputs. But when a human looks at what it’s actually recommending—especially in edge cases, in the long tail—there’s something off about it. Not catastrophically, not always. Just… off.

The system learned the pattern, including the pattern of your data quality problem.

Here’s what’s actually happening underneath. Your data has three types of defects:

Structural defects are easy. Missing values, wrong data types, things that break schemas. Your tools catch these. You fix them. Life goes on.

Semantic defects are harder. A field labeled “revenue” that means net revenue in one business unit and gross in another. A “customer type” that was coded one way three years ago and a different way now, but nobody updated the historical data. A date field that’s sometimes a transaction date and sometimes a creation date, depending on which system it came from. These are invisible to structural checks. They look clean on a data quality scorecard. And they teach your model nonsense.

Temporal defects are the ones that kill you. Your historical data is, by definition, old. It was collected under different business conditions, different processes, different market dynamics. An AI system that learns from it is learning from a past that no longer exists. You’ll discover this about six weeks after your system goes live, when its predictions start diverging from reality.

The real question isn’t: is our data good enough?

It’s: what decisions are we letting this system make, and what defects can we actually live with?

This is where most conversations break down. Because it forces you to be specific. Specific means uncomfortable.

A recommendation engine that occasionally suggests a slightly irrelevant product? You can live with that. Your customers see a mediocre recommendation, they ignore it. Cost of that failure: low.

A system that scores credit risk or flagged fraud, and it’s learned to be structurally inconsistent about what “fraud” means? You can’t live with that. You’ll catch it in compliance review three months in and have to rebuild it.

The practical move isn’t to announce you need “better data quality.” That’s governance theatre.

The practical move is to map your data defects against the specific decisions you’re making. Ask yourself: where does this data come from? Who was incentivized to enter it, and what were they incentivized to do? What changed three years ago in the system that collected this? Where is it inconsistent, and what does that inconsistency mean for what we’re trying to predict?

Then: deliberately reduce the scope of decisions you let the system make until the defects stop mattering.

You might keep the system from making binary decisions and make it advisory instead. You might restrict it to certain business units until you understand why the data behaves differently there. You might run it in shadow mode longer than you planned because the temporal defects are real and you want to see how it drifts.

This is boring. It’s not a feature announcement. It won’t make the quarterly exec summary.

But it’s the difference between a system that hums along in production for three years and one that breaks down in month eight when someone realizes it’s been quietly making recommendations that contradict its own logic because your semantic defects taught it two contradictory patterns.

Your data quality problem isn’t that you need better data. It’s that you need specific knowledge about where bad data actually matters, and you need the organization discipline to say: not yet. Not there. Not until we understand this.

That’s harder than buying a data quality tool. And it’s exactly the kind of hard that distinguishes deployments that compound value over time from the ones that become cautionary tales.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.