Why AI pilots stall before production

An AI pilot can work in a demonstration and still be unready for daily use. Before starting another one, check four things: whether the data preparation can be repeated, who will operate the system, how success will be measured and what it will take to run the work in its intended environment.

Where AI pilots stop

1. 01 Idea
2. 02 Demo
3. 03 It works
4. 04 Someone uses it The break
5. 05 It is theirs

The break falls between proving a thing works and anyone using it.

The uncomfortable part is that a stalled pilot usually looks like a success right up until it stops. The model performed. The demo landed. Everyone in the room agreed it was impressive. What did not happen was anybody changing how they work, and that gap is rarely visible in the final presentation.

The data was cleaned by hand

A pilot may begin with a prepared extract: duplicates removed, dates standardised and records joined by hand. That can be a sensible way to test an idea. The limitation appears when the production workflow needs the same preparation every day and nobody has built or assigned it.

Ask early what was done to the data before the model saw it, and ask whether that work is repeatable by a machine. If the answer involves a person and a spreadsheet, the pilot has not been tested against production conditions and the timeline is longer than it looks.

Nobody owns the Tuesday

A pilot has a project team. Production has an owner, which is a different thing. Somebody has to be accountable for the output being right on an ordinary Tuesday in March, for noticing when it drifts, and for deciding what happens when it is wrong.

Make the handover explicit before the project team disbands. The operating owner needs to agree what they are taking on, how they will notice a fault and who can help resolve it. An assumed handover is not an operating plan.

Success was never defined

Without a written threshold for continuing, a pilot’s result is easy to judge by the quality of its presentation. Decide what evidence would support a build, a change of approach or a stop before the work starts.

Write the threshold down before the work starts. Not a vague improvement, but a number against a current baseline, measured the same way both times. If the current baseline cannot be measured, that is the first piece of work, and it is worth knowing that before spending on a model.

The work lives somewhere you cannot reach

A pilot built in a supplier’s environment may need extra work before it can run in yours. Establish that work early. Check the code, dependencies, access, model artefacts, evaluation results and runbook, and test the transfer before treating deployment as a small final step.

The test worth applying is whether the work could continue if the supplier stopped answering the phone next week. It is an uncomfortable question to ask in month one and a much more expensive one to answer in month nine.

What to do before the next one

Two things carry most of the weight. Decide who owns the output in production before the pilot starts, by name, and write down the result that would justify going ahead. Everything else on this page is easier to fix once those two exist.

It is also worth being honest about what a pilot is for. A pilot that proves a thing is possible has done its job even if it never ships, as long as that was the stated purpose. The waste comes from pilots that were meant to reach production and were never set up to.

Write the next pilot brief in six lines

• The decision or task we want to improve:
• What happens today, and the baseline we can measure:
• The result that would justify continuing:
• The real data and environment the test will use:
• The person who will operate it and handle exceptions:
• What our team must receive if the work stops:

If one line cannot be completed, make resolving it part of the scope. That is useful work to do before building.

These are practical failure modes, not a ranking from a survey. How often each one occurs has not been measured here. A pilot that rules out an unsuitable approach can still be valuable if that was the agreed purpose and the learning is kept.