Most organizations make one mistake about AI governance: they treat all decisions as if they belong at the same level.

They create a governance board. The board makes decisions about which models to deploy, what risk thresholds are acceptable, how to manage fairness, what data to use. Everything flows through the same decision-making structure.

This works until it doesn’t. Suddenly the board is making decisions about whether a particular model should log timestamps in a particular format. Or whether a team should run a quick model experiment. Or what documentation a data scientist should write. And now the board is a bottleneck, because the board is making decisions that should be made much closer to the work.

The mistake is not that the board is doing anything wrong. The mistake is that the decision belongs at a different level.

The Three Levels

Most enterprise AI governance needs three levels of decision-making, with different questions at each level.

Enterprise level. These are decisions about strategy, architecture, and constraints that affect the whole organization.

Questions at this level: What’s our overall risk appetite for AI? What vendors do we work with? What data can be used for AI? What governance structures do we require? How much are we spending on AI? These are decisions that affect multiple lines of business, that have strategic implications, that set constraints others work within.

These decisions should move slowly. They should be made carefully. They should be made by people who can see across the whole organization. They should be made infrequently. If you’re revisiting your enterprise AI strategy every month, something is wrong.

Line-of-business level. These are decisions about which AI initiatives to pursue within a line of business, how to implement them, and how to measure success.

Questions at this level: Should we build a demand forecast model? Should we use this vendor’s AI platform or build our own? How do we resource the AI team? What’s the model refresh schedule? These are decisions that affect one business area but don’t affect strategy organization-wide. They’re guided by enterprise-level decisions (we will spend this much, we will use this vendor, we will follow this risk appetite) but they’re made locally.

These decisions should move faster. A line-of-business leader should be able to make the call on whether to pursue an initiative without going back to the enterprise board. If the enterprise board has set clear constraints, the local decisions are within those constraints. Move on.

Team level. These are decisions about how to build, test, and deploy models within an initiative.

Questions at this level: What features should we engineer? How many samples do we need for training? What’s the right batch size? Should we use this algorithm or that one? How should we handle missing data? These are implementation details. They’re guided by the line-of-business decision (we’re building a demand forecast) but they’re decided by the team that’s doing the work.

These decisions should move very fast. A data scientist should be able to choose an algorithm without asking permission. The team should be able to run experiments without a governance board vote.

The Common Mistakes

Organizations usually get this wrong in one of two ways:

First, they push decisions up. A team wants to run an experiment. They go to the board. The board approves it. This happens for every decision, so the board becomes a bottleneck. The organization moves slowly. Teams get frustrated.

This usually happens because the team isn’t confident in the constraints set at the higher level. They don’t know what they’re allowed to do, so they ask. Which means the enterprise level hasn’t been clear about what the rules are. Or the line-of-business level hasn’t been clear about the initiative. The failure is higher up, but it manifests as slowness lower down.

Second, they push decisions down. Teams start making decisions about architecture. Teams choose vendors. Teams set risk thresholds. And now you have no consistency across the organization. One line of business is using Vendor A, another is using Vendor B. One team is comfortable with 3% bias, another requires 0.5%. One team runs models without any governance, another has heavy governance.

This usually happens because the higher levels haven’t set clear constraints. Or haven’t been clear about which decisions they’re responsible for. The teams are making reasonable decisions locally, but those decisions don’t add up to a coherent strategy.

How to Tell If You Have It Right

You have the right decision hierarchy if:

Enterprise decisions feel weighty. When the enterprise board meets, they’re making decisions that feel consequential. Not “should we tune this hyperparameter?” Decisions like “what’s our risk tolerance?” or “should we use this vendor?” These should feel substantial because they’re the decisions that matter most.

Line-of-business decisions move fast. Business leaders can make the call on which initiatives to pursue without spending six months getting approval. They know the constraints (spend budget X, follow risk appetite Y, use vendor Z). They decide: we’re building a demand forecast, we’re building a churn prediction model, etc. They make those decisions independently.

Team decisions happen in hours or days. Data scientists run experiments. They try algorithms. They refine features. They don’t ask for permission. They work within the constraints set by the business leader. They make technical choices rapidly.

If enterprise decisions feel trivial, you’ve pushed decisions up. If team decisions require approval, you’ve not pushed decisions down. If different lines of business are making inconsistent choices about vendors or risk appetite, you haven’t set enterprise constraints clearly.

How to Implement It

If you don’t have this structure, start by answering one question: What decisions have to be made at the enterprise level to set clear constraints for everyone below?

For most organizations, the answer looks like: – Risk appetite (what are we willing to accept?) – Vendor strategy (what infrastructure do we use?) – Data governance (what data can be used for AI?) – Budget (how much do we spend on AI?) – Required processes (what governance do we require?)

Make those decisions carefully and clearly. Document them. Make them known. Then say: “Within these constraints, you can make your own decisions.”

The line-of-business level then works within those constraints: we have $X budget, we have to use vendor Z, we have to follow risk appetite Y. Now: what initiatives should we pursue?

The team level then works within the initiative: we’ve decided to build a demand forecast. Now: how do we build the best forecast possible?

Three levels. Three sets of questions. Three speeds.

The Conversation That Matters

The conversation worth having is: “Are we making this decision at the right level?”

If you’re at the board, and you’re deciding whether a team should use algorithm A or algorithm B, you’re at the wrong level. Or the team doesn’t have good constraints, and that’s a higher-level problem.

If you’re a business leader, and you don’t know whether you can pursue an initiative without asking the enterprise board, you need higher-level clarity about what the constraints are.

If you’re a data scientist, and you can’t run an experiment without approval, either the governance is too heavy, or your manager hasn’t given you clear guidance about what you can do.

The organization that gets this right doesn’t move faster in every dimension. It moves faster where it should (team decisions), and slower where it should (enterprise decisions). But because every level is moving at its appropriate speed, the overall system is faster and more coherent than an organization where everything goes through one decision-making structure.

Most organizations can implement this in a month. The hard part isn’t the structure. The hard part is being clear about what belongs at each level, and then having the discipline to stick to it.