The Beaver built the routing layer over a weekend, because he had noticed something on the Friday and could not leave it alone, which is the defining characteristic of the animal.

What he had noticed was that a substantial proportion of the requests coming through the Watering Hole were not questions at all. They were classification. Extraction. Reformatting. Deciding which of four categories a thing belonged to. Turning one shape of text into another shape of text.

All of it was going to the most capable thing available, because that was what had been wired in during the pilot, eighteen months earlier, by somebody who was trying to find out whether the idea worked at all and quite reasonably did not want the model to be the reason it failed.

Nobody had revisited it. Nobody revisits a working configuration.

By Monday, requests were being sorted by what they actually required. The bill fell substantially over the following weeks.

Nobody noticed for a month. Not one animal in the jungle reported a difference in output quality, because there was no difference in output quality, because none of the redirected work had needed the expensive path in the first place.

The Fox, when he found out, was pleased and then quietly irritated, and was honest enough to say so out loud. The largest single cost improvement of the quarter had come from plumbing, over a weekend, from a Beaver who had not been asked. It had not come from strategy, or governance, or any of the framework work that had consumed the previous two acts.

“Both things are true,” said the Crocodile, one eye open. “You needed the framework to know the plumbing mattered.”

The observation underneath it

Most enterprise AI traffic does not need the most capable model available. Not some of it. Most of it.

Once a capability is in production and serving real workflows, the traffic mix shifts heavily toward the mundane. Routing a request. Extracting fields. Classifying an input. Summarizing something short. Formatting an output. The genuinely hard reasoning that justified choosing a frontier model in the first place turns out to be a minority of the volume, sometimes a small one.

And every one of those mundane requests is being charged at the rate of the hardest thing your system does, because a default was set during a pilot and defaults are permanent.

This is the most fixable line on most AI bills and it is fixable in weeks rather than quarters. It is also the least glamorous thing in this entire series, which is precisely why it survives for eighteen months in most organizations.

Three routing patterns

Static tiering. The task type determines the model. Classification goes here, extraction goes there, open-ended reasoning goes to the expensive path. Simple, predictable, easy to reason about, easy to explain to a finance function, and it captures the large majority of the available saving. If you do nothing else from this post, do this.

Cascade. Try the cheap tier first. Escalate to the expensive one if some confidence or validation signal says the cheap answer was not good enough. Higher ceiling than static tiering and considerably more complexity, and it lives or dies on one number that I will come to.

Semantic routing. A classifier looks at the request and decides where it goes. Powerful, elegant, and the easiest of the three to over-engineer into something that costs more to operate than it saves. I would reach for this third rather than first, and only once static tiering has told you where the volume actually is.

The cascade math people get wrong

A cascade only saves money if the cheap tier resolves a high enough proportion of requests. Below that proportion, you are paying for two calls instead of one on the escalated portion, plus the added latency, and you have made things worse while feeling sophisticated.

The break-even depends on the price ratio between your tiers. The wider the gap between cheap and expensive, the lower the resolution rate you need to make it worthwhile. With a large ratio, a cheap tier that handles half the traffic is comfortably worth having. With a narrow ratio, you may need it to handle the great majority before the arrangement pays for itself at all.

Work out your own threshold from your own price ratio, then measure your actual escalation rate against it, and put both numbers on the same page.

I would guess that a meaningful share of deployed cascades in production today are sitting below their break-even and nobody has checked, because the cascade was built as an optimization and optimizations do not get audited. They get built, celebrated, and then assumed to be working forever.

Sufficiency, not maximization

The mental shift this requires is the hard part, and it is a business shift rather than a technical one.

The question is not which model is best. That question has a clear answer and it is the wrong question. The question is which model is sufficient for this task, at an acceptable failure rate, given what happens when it fails.

Sufficiency is defined by consequence. A misclassified internal document that a person reviews anyway has a low cost of error, so a high failure rate is tolerable and the cheap tier is fine. A misclassified input feeding an automated decision that reaches a customer has a high cost of error, and the expensive path is correct regardless of unit price.

That is a business judgment. Engineering cannot make it, and when engineering is forced to make it by default, it will reasonably choose the safest option every time, which is how you end up where the jungle was.

You cannot route without evaluation

The dependency that stops most organizations doing any of this.

You cannot route by sufficiency unless you can measure sufficiency. Without an evaluation harness that tells you how each tier performs on each task type, routing is guessing with extra infrastructure, and the first time somebody complains about quality the whole thing gets reverted.

This is the evaluation layer from Lesson 3, the one that appears on nobody’s budget. It turns out to be the precondition for the largest cost lever available. Which is a fairly typical shape in this discipline: the unglamorous investment nobody wanted to fund is what unlocks the saving everybody wants.

Why this belongs at the gateway

Routing must live at a shared gateway rather than inside each application, and this is the architectural decision with the longest tail in the whole series.

If each team makes its own model choice inside its own code, then every choice becomes a hard dependency. Changing tier later means touching forty codebases owned by forty teams with forty roadmaps, which means you will not change tier later, which means every routing decision made this year is permanent.

Put it at the gateway and the tier becomes configuration. You can change it centrally, test it, roll it back, and adjust it every quarter as prices and capabilities move, which they will.

The gateway is also where the tagging from Lesson 1 lives, where the allocation from Lesson 7 becomes possible, and where the guardrails in Lesson 19 will need to sit. It is the single highest-leverage piece of architecture in an enterprise AI estate and it is almost always built late, after the pain, by someone doing a migration.

Three ways this goes wrong

Top tier by default. The pilot configuration, still in production, quietly handling string formatting at premium rates for two years. This is the jungle’s situation and I would expect it to be more common than not.

The negative cascade. Escalation rate below break-even, so the clever thing costs more than the naive thing, and nobody has calculated the threshold to find out.

Routing in the application. Every team choosing independently, every choice becoming permanent, and a tier change turning into a migration project that never gets prioritized.

The Field Kit

Concrete things to do this week.

If you sit in the Crow’s chair, ask two numbers. What share of requests go to your most expensive tier, and what share genuinely require it. The gap between those is money, and it is usually available within a quarter.

If you sit in the Crocodile’s chair, get routing behind a shared gateway before you have forty callers. Retrofitting this is the most avoidable expensive project in the discipline and the window to avoid it closes quietly.

If you sit in the Mandrill’s chair, define the quality floor for your use case, expressed as an acceptable failure rate and what happens when it fails. Sufficiency is a business judgment and if you do not make it, somebody will default to the most expensive option on your behalf, forever.

For everyone: if you run a cascade, calculate its break-even and measure your escalation rate against it. Today. It is a ten minute calculation and there is a real chance it tells you something unwelcome.

Jungle Lesson 12

Most of what you send to the cleverest animal in the jungle does not need the cleverest animal in the jungle. Define what sufficient looks like for each task and buy down to it, because paying for judgment you are not using is the most common and most fixable line on the bill.

Next time: the Beaver takes the Crow back to the river he mentioned all the way back in Lesson 1 and shows her what it is actually carrying. It turns out you are billed for the question as well as the answer, that in most enterprise workloads the question is the larger number, and that every line somebody added to a prompt after an incident two years ago is still being paid for on every single request. Lesson 13 is about context.

The Beaver in your organization has probably already noticed the thing in this post. Whether anyone has asked him is a different question, and it is usually the answer to why it has not been fixed.