The golden set is a spreadsheet with about two hundred rows. Each row has an input taken from real usage, the output a competent human produced for it, and a short note on why that output is right. It is the least impressive artifact this company has ever produced. Nobody has ever asked to see it in a meeting.

It is also the only reason we found out, on a Tuesday in the middle of an otherwise ordinary month, that the product had been getting worse for eleven days.

Nobody had deployed anything. Nothing had broken. The model underneath had been updated, quality on one category of task had moved, and every dashboard we had was green because every dashboard we had measured uptime, latency and error rates. All three were excellent. The product was worse and the instrumentation had no opinion about it.

A problem with no equivalent in ordinary software

Conventional software is deterministic. The same input produces the same output, so a regression is a failing test and a failing test is a red build. The whole discipline of continuous integration rests on that property.

An AI product does not have it. The same input produces different outputs. Worse is not a boolean, it is a movement in a distribution, and you cannot detect a movement in a distribution without a reference point.

There is a second difference which is stranger and which founders reliably underestimate. Your product can change without you changing it. The capability you built on is maintained by somebody else, on their schedule, for their reasons. A model update can improve nine things and degrade the tenth, and the tenth might be the one your customers actually bought.

In ordinary software, the ground does not move. Here it does, and you need an instrument that notices.

What a golden set actually is

One hundred to three hundred real cases, drawn from real usage, each with an output that somebody competent has agreed is correct.

Four things make the difference between a useful one and a decorative one.

It is built by the person who does the job, not by an engineer. The engineer knows what the system does. The practitioner knows what right looks like, and those are different pieces of knowledge. If you have design partners from Lesson 6, this is one of the most valuable things to ask them for, and it is a reasonable thing to pay for.

It contains the hard cases deliberately. A set made of representative examples tells you about the average, and the average was never the problem. Include the ambiguous inputs, the malformed ones, the ones where two answers are defensible, and the ones where the correct behavior is to refuse or escalate. That last category is the one most sets omit and the one that matters most for an accountable product.

It is versioned like a product asset. Not a file in a test directory. It has an owner, it grows when incidents reveal a gap, and every entry records who decided and when.

It is small enough to run constantly. Two hundred cases you run on every change beats two thousand you run quarterly. The value is in the frequency.

Three gates, on every change

Once the set exists, it becomes a gate, and it should sit in front of three questions rather than one.

Quality. Score the change against the golden set. Block on any regression beyond an agreed threshold. Not a warning in a log, a block.

Cost. Compute cost per query for the new path and compare it against the current one before merging. A feature that improves quality by two percent and raises cost per query by four hundred percent is a decision somebody should make consciously. Without this gate, that decision gets made by an engineer at eleven at night and discovered by you in an invoice.

Latency. The same discipline. A change that adds two seconds is a product change even if nobody wrote a product ticket.

The critical scope point: these gates run on changes to prompts, retrieval configuration and model selection, not only on changes to application code. A prompt is code. It is the most frequently edited and least reviewed code in most AI companies, and it is routinely modified directly in production environments by people who would never do the same to a function.

Scoring without pretending

Exact matching works for extraction and classification. It does not work for anything generative, and pretending otherwise produces an evaluation that punishes correct answers for being phrased differently.

The practical approach is rubric scoring with a model acting as judge: define what a good answer contains, have a model grade against that rubric, aggregate. It is fast, it is cheap, and it works well enough to be genuinely useful.

It is also biased in ways that are easy to forget. Models grading models tend to reward fluency, length and confidence, which are three properties of exactly the output described in Lesson 12. So two safeguards are not optional. Sample a fixed percentage for human review every cycle and budget the hours properly. And track the rate at which the humans disagree with the judge. That disagreement rate is your instrument for whether your instrument still works, and when it drifts you have an evaluation problem rather than a product problem.

Pin your versions

Two habits close the loop on the eleven day problem.

Pin explicit model versions rather than accepting whatever the latest alias resolves to. Then, when a new version appears, run the golden set against it, look at cost and latency alongside quality, and adopt it as a deliberate change with a date attached.

And log the model version, prompt version and retrieval configuration with every single request. When a customer says it used to do this correctly, that logging is the difference between a two-hour investigation and a fortnight of theories.

Building this before the demo feels wrong. The demo is what gets you the meeting. But the demo is a claim, and the harness is the only thing that lets you keep making the claim after the fourth model update, the second retrieval rewrite and the day you route half your traffic to a cheaper tier to protect your margin.

If you cannot detect a regression, you do not have a product. You have a demo that has not broken yet.

Tomorrow: the architecture diagram, and the question of what you would have to rebuild on Monday.