Fifteen lessons for the person running an enterprise AI evaluation, published every working day from 27 August to 16 September 2026.

It is written from the buyer’s chair, by someone who has run these evaluations rather than someone selling into them. Every lesson is built on one anonymized composite, merged from several real evaluations so that no single organization is recognizable, and arranged in the order the problems actually arrive: what you settle before you talk to anyone, what happens during evaluation and pricing, then the contract and everything after it.

No vendor or product is named anywhere in the series. That is deliberate. The structural points hold regardless of whose logo is on the proposal, and a series that named names would date within a quarter.

Every figure is illustrative and is marked as such in the sentence that carries it. Every lesson ends with one thing to do differently, which is the part worth keeping. None of it is legal, tax or procurement advice.

Part I: Before You Talk to Anyone

The decision you are actually making, the problem you can defend, and everybody who can stop you.

• Lesson 1: The Chair You Are Sitting In (27 August). You are not being asked whether to buy, only which to buy. The two things you know that the vendor cannot are worth more than they look, and most buyers give both away in the first two meetings.
• Lesson 2: The Problem You Can Actually Name (28 August). A vaguely stated requirement cannot discriminate between vendors, so something else decides. Usually the demo.
• Lesson 3: Build, Buy, or Wait (31 August). Waiting is the only option that gets cheaper while you think about it, and the only one nobody costs.
• Lesson 4: The Business Case You Will Defend in Eighteen Months (1 September). The case is written for the approval meeting and read properly once, much later, by somebody with no stake in the answer being flattering.
• Lesson 5: Who Is in the Room (2 September). Approval maps are not power maps. Purchases die in the gaps, and the two people who can stop you will never say no.

Part II: Evaluation and Price

Pilots that can fail, demos that are products, references that are samples, and what sits under the quote.

• Lesson 6: Pilot Theatre (3 September). Pilots almost never fail, which is the tell. A proof of concept without a kill criterion agreed in advance is a procurement formality with a demo attached.
• Lesson 7: The Demo Is a Product Too (4 September). The demo has its own backlog and its own maintainer. Three requests show you what it was built to leave out.
• Lesson 8: Reference Calls That Get Real Answers (7 September). Three selected references sample one population. Ask what they would put in the contract if they were signing again.
• Lesson 9: Reading a Cost Model From the Outside (8 September). You cannot see a vendor’s costs, but the four layers under the quote behave predictably, and what they defend tells you where their money goes.
• Lesson 10: Seats, Tokens, Outcomes (9 September). Every pricing shape creates an incentive that works against the reason you bought the thing. Pick the version you would rather manage.

Part III: The Deal and After

Diligence, contracting for a moving target, leaving, and the renewal that is the real negotiation.

• Lesson 11: Diligence That Is Not a Questionnaire (10 September). Your security questionnaire tells you almost nothing. Five questions replace it.
• Lesson 12: Contracting for Something That Changes Under You (11 September). Model versioning, deprecation notice, and behavior that changes without a single term being amended.
• Lesson 13: Exit Rights and the Leave Test (14 September). We can always switch is a claim, not a plan. Test it before you sign, not at renewal.
• Lesson 14: Renewal Is the Real Negotiation (15 September). All of your leverage is in month ten, and most buyers spend it in month one.
• Lesson 15: You Are Building a Capability, Not Running a Purchase (16 September). The evaluation ends. What you should be left with is an organization better at doing this than it was in August.

The pattern across all fifteen

Nearly every failure in this series comes from the same place: a question that was cheap to ask early and expensive to ask late. What would make us say no. What does this look like when it is wrong. Which parts of this price move if our usage doubles. Who could stop this without ever refusing.

None of those require special expertise. They require asking while the answer can still change something, which is a matter of sequence rather than skill, and sequence is the one thing a buyer fully controls.

Lessons link from here as they publish. The full archive lives under In Practice: The Other Side of the Table.

This is one of five series. The others are In Practice: AI in the Enterprise, In Practice: Building an AI Company, Scaling AI FinOps in the Enterprise Jungle and While We Were Arguing. All five are listed on the series page.