You have agreed a pilot. Ten weeks, two vendors, a defined scope, a steering group, and nowhere in any of the documents is a sentence describing the result that would make you stop.

Pilots almost never fail. That is the tell. If your organization has run six of these and all six succeeded, the pilot is not measuring anything, it’s producing a decision that was already made and giving everyone something to cite.

The criteria that cannot produce a no

Composite from a few clinical deployments: a hospital group, several sites, a discharge documentation problem. The pilot ran ten weeks with forty clinicians.

The success criteria, as written, were to demonstrate feasibility and gather user feedback. Both of those are guaranteed before anyone starts. Feasibility was demonstrated, in that the system produced discharge summaries. Feedback was gathered, in that people had opinions. Illustratively, sixty-eight percent of participating clinicians said they would use it again, and that number became the headline in the steering pack.

What nobody measured was how heavily the drafts were edited, and whether editing took longer than writing from scratch on the complex cases. When someone finally looked, roughly the hardest fifteen percent of discharges took longer with the tool than without it. Those are also the ones where errors matter most.

The product was fine. It probably was worth buying, scoped differently. But the pilot had been built to produce a yes, and it produced one, and the scoping conversation that should have happened at week ten happened at month fourteen instead.

Four reasons your pilot will pass

None of these involve anyone behaving badly. They’re structural, which is what makes them reliable.

The vendor is staffing it. There’s a solution engineer in your channel answering within the hour. That is not the support model you’ll have in production, and it is quietly fixing problems you never find out existed.

Volunteers, not conscripts. The forty people who signed up are not a sample of the four hundred who’ll be told to use it in March. They’re the ones who like new tools. Their enthusiasm is real and it does not generalize.

The easy slice. Ten weeks is not long enough to assemble messy data, so pilots run on the clean subset. The clean subset is not where your cost is.

Accumulated commitment. By week ten the organization has spent four months, three workshops and a steering group’s attention. Saying no at that point costs someone social capital, and everyone in the room can feel it.

Building one that can fail

A pilot that can fail is not a hostile pilot. It’s the only kind that tells you something you didn’t already believe. Six things make the difference:

• One number, measured the same way before and during. Not a dashboard. One.
• A written kill criterion, agreed by the sponsor before the vendor is told the pilot is happening. If it’s written after, it will be written to be passable.
• Reluctant users in the population. Pick five people who didn’t volunteer and ask them to take part anyway. Their experience is the one that predicts month nine.
• The hard cases in scope. Name the ugliest ten percent explicitly and require them to be included, because that’s where the business case lives or dies.
• A declared support level. Ask what this looks like without a solution engineer in the room, then run at least the last three weeks that way.
• A named person who can stop it. Not a committee. One person, told in advance that stopping is an acceptable outcome and will not be held against them.

That last one costs nothing and is skipped almost every time. A committee cannot stop a pilot. Committees produce continuations with caveats.

The exit nobody writes down

Before the pilot starts, settle three things in writing. They take a paragraph each and they’re painful to negotiate afterward.

What happens to your data when the pilot ends, including anything derived from it. Whether outputs produced during the pilot can be used if you don’t proceed. And whether the pilot fee, if there is one, credits against a contract, because a pilot fee that only credits on signature is a small, entirely legal incentive pointed at your own decision.

The outcome your process cannot record

Sometimes the honest pilot result is that the thing works and isn’t worth the change cost. It does what it says, the number moves a little, and getting four hundred people to alter how they work costs more than the gain.

That is a successful pilot. It is also a failed purchase, and most governance processes have no box for it. The options are proceed, or proceed with conditions, or defer, and defer is where good projects go to be quietly embarrassing.

If you can add one box to your template this year, add that one. Works, not worth it. I’ve watched a room spend forty minutes trying to phrase that conclusion in a way that wouldn’t look like failure, and land on deferring pending further review, which is how a clear finding turns into eight months of nothing.

One thing to do differently

Before the pilot starts, write the single sentence that would end it. Then get the sponsor to sign that sentence, not the pilot plan.

The signature is the part that matters. It converts stopping from an act of individual courage in week ten into a thing that was agreed in week one, by someone senior, when it was still cheap to agree to.