The Beaver offered to show the Crow around the Watering Hole, and she accepted, mostly because she had approved funding for it twice and had never actually seen it.

Beavers cannot hear running water without building something. This is not a character flaw, it is a compulsion, and it is the reason the Watering Hole worked at all. It is also the reason it had considerably more moving parts than anyone in the Canopy realized.

He started with the part she knew about. “This is where the thinking happens,” he said. “This is the bit everyone means when they say the word.”

“Right,” said the Crow. “That’s the line in my budget.”

“That’s about forty percent of it.”

She stopped walking.

He opened a panel she had not noticed. Behind it, something was running steadily, working through the jungle’s documents, converting them into a form the system could search. It had been running all night. It ran every night.

“What is that costing us?”

“Depends how much anyone wrote this week.”

“Not how much anyone used it?”

“No,” said the Beaver. “That one doesn’t care whether anybody uses it.”

The Crow, who had spent a career learning that costs follow activity, took a moment with that. Then she asked him to open every other panel.

The layer you budgeted

Almost every organization budgets for inference and is surprised by everything else. Not because the other layers are hidden, exactly, but because the mental model arrived from somewhere else. You buy a thinking machine, you pay per thought. That is the shape of the thing as it is described in every article, every business case, and every conversation with a vendor.

The shape is wrong. Inference is usually somewhere between a third and half of the true cost of a running capability, and the proportion falls as the capability matures, because the supporting layers keep growing while the thinking gets cheaper.

Here is the full stack, in roughly the order it surprises people.

The seven layers

• Inference. The one you budgeted. Scales with usage, drops in unit price over time, and is the only layer most organizations can report on.
• Retrieval and indexing. Embedding your content, storing it, and re-processing it every time it changes. This scales with the size of your corpus and the rate at which people write things, not with how much anyone uses the system.
• Orchestration and glue. Queues, state, workflow, the compute that coordinates rather than thinks. Individually cheap, structurally permanent, and it never appears on a slide.
• Evaluation. Running your test sets, regression checks and scoring. Scales with release frequency, which means it grows precisely as the team gets better at shipping.
• Observability and logging. Capturing what went in and what came out, at volume, and keeping it for as long as somebody with an armored shell says you have to.
• Human in the loop. Review, correction, escalation, the person who checks the output before it goes anywhere consequential. This is a labor cost and it usually lands in a completely different budget.
• Rework. Output that was wrong, redone, or quietly discarded. Costs you twice and appears nowhere at all.

Seven layers. Most organizations can see one of them clearly, two of them vaguely, and are actively surprised by the rest.

The two that grow when nobody is using it

Retrieval and evaluation deserve their own paragraph, because they break the intuition that cost follows activity, and that intuition is the load-bearing wall of every cost conversation your finance team has ever had.

Retrieval cost tracks your content, not your usage. If a content team has a productive quarter and publishes four hundred new documents, your retrieval bill goes up, and not one additional animal has asked the system a single question. If somebody reorganizes a document library, everything gets reprocessed. If a well-intentioned team decides to index a large archive that nobody has opened since the last reorganization, you will pay to keep that archive current forever.

Evaluation cost tracks your release cadence. A team that ships weekly runs its test suites weekly. As the suites grow, which they should, the cost of every release grows with them. This is the correct behavior and it is still a cost line that nobody forecast, and it arrives at exactly the moment when a program is going well, which is the worst moment for a finance surprise.

Neither of these responds to a usage cap. Neither of these shows up in a conversation about adoption. Both of them appear on the invoice.

The layer in someone else’s budget

Human review is the one I see cause the most damage, and it is not close.

The capability produces output. Somebody checks it. That checking is real work performed by real people, and in almost every organization I have seen, those people sit in an operations budget, a service budget, or a shared function that has nothing to do with the AI program.

So the capability looks cheap. Its cost line contains inference and some infrastructure. The review labor it generates is invisible to it, sits somewhere else, and grows in direct proportion to how much the capability is used.

Then somebody decides to scale it, on the basis of a unit cost that excluded most of the marginal cost, and the operations budget quietly absorbs the difference until it cannot, which is usually about two quarters later and always somebody else’s problem to explain.

The Mandrill, when this was pointed out to him, did something I have a lot of respect for. He asked for the review hours to be attributed to his capability, knowing it would make his own business case look worse. It did make it look worse. It also made it real, and the version of the decision he made afterwards was a better one.

The invisible layer

Rework is the seventh layer and the one nobody can measure, which is precisely why it is worth naming.

Output that was wrong and had to be redone. Output that was fine and got discarded because the person did not trust it. Attempts that succeeded on the third try. Work generated, reviewed, rejected, and generated again.

You cannot instrument this precisely. What you can do is stop pretending it is zero, because zero is the number currently in every business case in your organization. A documented estimate, even a rough one, is a better input than an implicit assumption that the failure rate is nil. And the act of estimating it tends to surface how little anybody knows about acceptance rates, which is a useful thing to discover in a planning meeting rather than in an audit.

Fixed and variable

The last thing worth pulling out of the stack is which parts are fixed and which are variable, because that ratio determines the shape of every architecture decision you will make later.

Inference is variable. Human review is variable. Retrieval and evaluation are mostly fixed with respect to usage and variable with respect to other things entirely. Orchestration and observability are largely fixed once built.

A capability with a high fixed proportion has terrible unit economics at low volume and excellent unit economics at high volume. A capability that is almost entirely variable has flat unit economics forever. Neither is better. But if you do not know which one you have, you cannot answer the question of whether scaling it will help or hurt, and that question is going to come up.

One thing the Beaver said as we finished the tour has stayed with me. “Everyone asks me how much a question costs,” he said. “Nobody asks me how much it costs to be ready for one.”

Three ways this goes wrong

Inference tunnel vision. Months spent optimizing the visible layer while the other six grow unwatched. The optimization work is real and the savings are real and the total bill goes up anyway, which does considerable damage to the credibility of everyone involved.

Corpus creep. Retrieval costs rising steadily with no change in usage, and nobody connecting it to the fact that three teams have been busy publishing. This one can run for a year before anybody asks the right question.

The orphan labor line. Human review costs sitting in an operations budget, so the capability appears far cheaper than it is, and gets scaled on a unit cost that was never true.

The Field Kit

Concrete things to do this week.

If you sit in the Crow’s chair, pick one capability and demand the cost broken into all seven layers. Just one. You do not need the whole estate to make the point, and the exercise of producing it will teach the team more than the answer teaches you.

If you sit in the Crocodile’s chair, put retrieval and evaluation onto their own cost lines today, separately from inference. They are the two that grow silently and they are invisible inside an aggregate.

If you sit in the Mandrill’s chair, find the human review hours your capability generates and insist they are attributed to it. Yes, it makes your numbers worse. It also means the decision you make next is based on something true, and you will be the only person in the room who can say that.

For everyone: put a number on rework, even a guess, and write down how you arrived at it. Any documented estimate beats the zero that is currently sitting there by default.

Jungle Lesson 3

Inference is the layer you budgeted and rarely the layer that hurts. The expensive parts of an AI capability are the ones that grow when nobody is using it, and the ones that get paid for out of somebody else’s budget.

Next time: the Tortoise puts a list on the table. She has been quietly compiling it for two seasons and it is considerably longer than anyone expected, and several entries turn out to have been started by the Fox’s own team. Lesson 4 is about shadow AI, why the instinct to stamp it out is the most expensive available response, and why that list is the best product roadmap in the building.

If anyone has ever shown you a full seven layer breakdown of a live capability without being asked, keep them. They are rarer than they should be.