I have been handed exactly one training ROI calculation that I believed. It came from a maintenance manager, it fitted on half a page, and it said: we spent R380 000 training fourteen fitters on precision alignment, our coupling-related failures on the reagent pumps dropped from nine to two in the following year, and each of those events cost us roughly R210 000 in downtime and parts.
No control group, no discounting, no isolation of confounding factors. An academic would tear it apart. It was still the most useful evaluation document I saw that year, because the operations director agreed the measure beforehand and accepted the result afterwards.
Why the sophisticated models fail
The Kirkpatrick-Phillips lineage is intellectually sound and practically unworkable in most operating businesses. To get a defensible financial figure you need to isolate the training effect from everything else that moved — a new supervisor, a plant modification, a change in ore hardness, a different contractor.
Teams that attempt full isolation typically spend months on it and then, in my experience, do not publish, because the confidence intervals are embarrassing. So the organisation ends up with no evaluation at all, which is worse than a rough one.
The trap is treating measurement as a scientific problem when it is a governance problem. You are not proving causation to a journal. You are giving a decision-maker enough basis to fund the next round or stop.
Agree the indicator before you start
This is the entire discipline, and almost nobody does it. Before the programme is approved, write one sentence: if this works, we expect [specific operational number] to move from [current] to [target] by [date].
Examples I have used:
- Percentage of work orders closed with a valid coded failure mode, from 41% to above 80% within six months
- Number of technicians authorised for unsupervised MV switching on the eastern network, from 4 to 9 by December
- Repeat audit findings on document control, from 6 to 0 at the next external audit
- Mean time to isolate a feeder fault, from 47 minutes to under 30
Notice that none of these are rand figures. Convert to money only where the unit cost is already agreed and uncontested. If your finance team disputes the cost of an hour of downtime, do not build your case on it.
The four questions worth answering
Rather than five levels, I use four questions and keep the whole evaluation to a page:
Did the right people attend? Not how many. Whether the specific individuals whose capability gap drove the investment were in the room. Often they were not, because someone had a shutdown and sent a substitute.
Can they demonstrate it? A practical assessment against defined criteria, ideally by the line rather than the provider. This is the level most organisations skip and it is the cheapest one to run properly.
Are they doing it? Observed at 60 to 90 days, by the supervisor, against two or three specific behaviours agreed upfront. Ten minutes per person.
Did the number move? The single indicator, reviewed at the agreed date, in the operational forum where that number normally lives — not in an HR report.
What to do when the number does not move
Resist the urge to bury it. In three cases I can think of, the indicator did not move and the diagnosis was genuinely useful: in one, the delegates were competent but the spares availability made the new practice impossible; in another, the supervisor had never been briefed and actively discouraged the new approach; in the third, we had trained the wrong population entirely.
All three findings were worth more than the training. None would have surfaced without a pre-agreed measure and someone willing to report a miss.
A note on the annual report
Training days per employee and rand spent per head will continue to appear in integrated reports because B-BBEE scorecards and stakeholders expect them. Fine. Report them. Just do not confuse them with evidence that anything works — they are input measures, and an organisation can maximise both while getting steadily less capable.