Attendance is not impact.

A donor agency runs a programme. The programme runs 240 workshops in a year. 6,400 SMEs attend at least one. 4,100 complete the eight-session curriculum. The post-programme survey returns a 4.3 out of 5 satisfaction score. The annual report writes itself.

Three years later, the programme is renewed. The donor asks the question every donor eventually asks: are the businesses we supported measurably stronger than the ones we did not? The programme team cannot answer in evidence at the operator level. They have attendance, completion, and a satisfaction score. None of those tell anyone what changed.

What the evaluation literature actually shows

This is not a niche complaint. J-PAL's reviews of business-training programmes have synthesised randomised evaluations across multiple countries - Mexico, Peru, Tanzania, Sri Lanka, the Philippines, and others. The consistent finding is that the average measurable effect of business training on revenue, employment, and business survival is small or null. Mentoring interventions and finance-plus-light-training combinations perform somewhat better in some contexts. Pure classroom training generally does not.

The World Bank's reviews of entrepreneurship programmes reach broadly similar conclusions. The Inter-American Development Bank's evaluations of business-development services in Latin America repeat the pattern. The OECD's annual Financing SMEs and Entrepreneurs Scoreboard, tracking roughly fifty countries, documents that the data is rich at the credit-condition layer and sparse at the intervention-outcome layer.

The conclusion is not that training does not work. The conclusion the methodological caveats in those reviews keep returning to is that programmes measured against incompatible rubrics produce noisy aggregate estimates. The sector cannot distinguish the interventions that worked from the ones that did not - at the operator level, across countries - because no shared operator-level evidence layer exists.

The metrics we built around what was easy to count

Attendance is easy to count because it is exposed by the operating system of every programme. So is completion. So is satisfaction. These metrics survived because the cost of capturing them is low and the appearance of impact is high. What they cost in reporting credibility is significant.

The IFC, OECD, World Bank, and major DFIs have all converged on the same recommendation in different language: SME-development outcomes need to be measured at the operator level, with comparable evidence rubrics, across programmes and countries. The recommendation has been published for over a decade. The sector-wide instrument that would let programmes operationalise it has not been built and published in the open.

A programme can show 4,100 completions and not be able to name a single business that grew in a measurable way. Attendance is necessary. It is not sufficient.

What measurable impact looks like at the operator level

A serious impact metric describes movement of a business from one verifiable state to another. The five dimensions worth measuring at programme close compared to baseline are not new - they have been discussed in development-evaluation methodology since the early 2010s. They have not been operationalised at sector scale because no shared instrument has named them at the operator level.

None of these dimensions requires new measurement technology. Conversational AI intake, document parsing, banking API integration, and structured evidence vaults are operational technologies available today. What has been missing is one instrumentation layer that connects them at the operator level and reports on movement in language that institutional accountability frameworks recognise.

The cost of getting this right

Programme teams that move to evidence-based reporting pay a cost at programme design time. They have to be explicit about which stage transition, which levers, which evidence outputs the programme commits to. They cannot promise "skills uplift" without defining what skill, what level, what evidence.

They pay a smaller cost during programme delivery. The structured intake conversation that produces the baseline is itself useful to the operator. The interventions paced through the programme are now matched to specific gaps. The completion event is the evidence pack, not a certificate.

They benefit at programme renewal. The board, donor, or minister reviewing the next cycle does not need to take attendance metrics on faith. They can see, per operator, what changed. Programmes that report this way tend to keep getting funded. Programmes that cannot answer "did the businesses we supported get measurably stronger" tend not to.

Where this leads

The instrumentation layer that allows attendance to become evidence, completion to become movement, and satisfaction to become signal-of-discipline is not theoretical. It is the work the Mothusi Growth Score is built to provide. The published methodology and the open evidence schema are visible to anyone who wants to review them.