There is a moment, early in AI adoption, when a decision gets made that looks small at the time.

It’s not a decision about which model to use. It’s not about governance structure or hiring. It’s about architecture. Usually it gets made by a technical team, noted in a meeting, maybe documented somewhere. The decision is: Where do we put this model? Do we build it ourselves, or do we use the vendor’s platform?

This decision will cost you millions.

Most organizations make it wrong, without realizing it’s the decision they’re making at all. They start using a vendor’s AI platform—whether it’s a CRM provider’s AI features, a data platform’s ML tools, or cloud AI services from a hyperscaler. It’s the easy path. The vendor handles the infrastructure. The model gets deployed in weeks instead of months. You’re running. You look like you’re moving fast.

Then, five years later, something changes. You’ve built your business around that vendor’s model serving layer. Your data pipelines feed into their architecture. Your applications are integrated with their APIs. Your teams know their tools. And you realize you’re locked in. You want to move to a different model, or a different vendor. It’s technically possible. It’s just expensive. Maybe hundreds of millions expensive, depending on your scale.

You’ve created technical debt so deep that switching isn’t a business decision anymore—it’s a strategic bet.

How This Happens (Without Anyone Meaning For It To)

The reason this happens is not malice. It’s not even vendor lock-in in the traditional sense. It’s compounding convenience.

Day one, you adopt a vendor’s platform because they have the best model for your use case. Smart decision. Day 30, you wire your data into their infrastructure because that’s where the model lives. Makes sense. Day 60, you build a second model on the same platform because you’re already there, and moving the first one would be chaos. Still reasonable.

By month eighteen, you have seventeen models running on that vendor’s infrastructure. Your data team knows the vendor’s APIs. Your DevOps team understands their monitoring. Your compliance team understands their audit trail. You’ve solved all the hard problems using their tools. The cost of moving is not the cost of migrating one model. It’s the cost of rebuilding your entire AI operating system on new infrastructure.

And the vendor knows this. Not because they’re sinister. Because they’ve seen it a hundred times. They don’t need to lock you in. They just need to be the default until it costs too much to change.

What This Costs

The most visible cost is money. If you’ve built on your data platform’s vector search and you want to switch to a different vector store, or if you’ve built on your CRM vendor’s prediction serving and you want to move to open-source, you’re rewriting years of integration. You’re retraining your team. You’re building new monitoring, new audit systems, new deployment pipelines.

At enterprise scale, this is eight figures. Sometimes nine.

The less obvious cost is velocity. During the transition—which takes a year or more—you can’t update your AI infrastructure confidently. You can’t adopt new techniques because you don’t know if they’ll work in both the old and new systems. You’re operating in a carefully managed state of semi-paralysis. Your competitors, who made different architecture decisions, are moving faster.

The strategic cost is control. If a fundamental shift in the AI landscape happens—if there’s a new model family that’s meaningfully better, or a new architecture pattern that changes the economics of AI, or a regulatory requirement that changes how you need to serve predictions—you have to evaluate whether it’s worth moving. And you usually conclude it’s not, because the switching cost is too high. So you stay on a legacy architecture longer than you should.

The Decision That Matters

Here’s what this looks like as a decision: Do you build an abstraction layer between your applications and the model serving infrastructure?

If you do, switching vendors costs the same as switching databases. Annoying, but doable. Your applications talk to your abstraction layer. Your abstraction layer talks to any inference infrastructure. You can swap the infrastructure underneath without touching application code.

If you don’t, your applications talk directly to the vendor’s serving layer. Switching costs much more, because now application teams have to learn new APIs, integrate new authentication, handle new failure modes.

The abstraction layer costs more up front. You’re not using the vendor’s native optimization. You’re solving problems twice. The vendor does most of the work for you in their platform; with an abstraction layer, you’re doing that work yourself.

Most teams skip it because of this cost. It’s the wrong calculation.

What To Do About It

If you’re starting your AI program now, here’s the straightforward version:

Build your own model serving layer, even if it’s thin. Or use an open-source serving platform (vLLM, Ray Serve, BentoML, TensorFlow Serving) and commit to it. Wire your applications to talk to your serving layer, not the vendor’s. Use the vendor’s platform for fine-tuning, for training, for data preparation. Don’t use it for serving.

The cost is real—maybe 15% slower inference, maybe more engineering overhead. But the switching cost if you need to move drops from nine figures to one. Your ability to adopt new models without a five-year migration project improves dramatically.

If you’re already locked in—if you’ve already built seventeen models on one vendor’s infrastructure—here’s the harder conversation: Accept that you’re going to stay there for a while, but start building an abstraction layer for new models. Don’t migrate the old ones. But new models go into the abstraction layer first. In five years, you’re a hybrid. In ten, you could switch.

Why This Matters To You

Most boards don’t think about architecture. Architecture is a technical detail. But architecture creates lock-in, and lock-in creates cost, and cost constrains strategy.

When you’re evaluating AI spending in five years, and it looks higher than you expected, and someone asks why you can’t just switch to a cheaper model serving platform, the answer is going to be “because of a decision we made in 2026 and never revisited.”

This decision doesn’t need to be made by a board. But it should be made consciously, by someone who understands that convenience today is strategy tomorrow.