What to measure, when, and how?
The above question can be broken down into smaller ones: what do we need to measure in terms of effectiveness and impact? When do we need to take those measures? And how do we measure them? Let’s tackle those questions in order.
What do we measure?
The most familiar model of measuring learning is still the one developed by Kirkpatrick in the 1950s and updated by Jack Philips in the 1980s, although it is most effective when applied using Wiggins and McTighe’s Understanding by Design approach. This provides five possible levels of evaluation that can be used:
- Return on expectation (RoE) – has the training has met the stakeholders’ expectations?
- Participant reaction – have suitable conditions been created for learning to take place?
- Measure of learning – did learning actually take place?
- Job impact – did the training influence the learners’ on-the-job performance?
- Return on investment (RoI) – has the financial payoff in terms of upskill staff, improving professional performance, and meeting strategic measures and priorities justified the expense of creating the training in the first place?
A comprehensive evaluation of training effectiveness involves appraising impact at all five stages. However, it is equally possible to select the measures that are most relevant to your organisation’s priorities and interests. There are a variety of other training evaluation models available too, and it is possible combine approaches create a bespoke picture that is best suited to your organisations needs.
A word of caution is worth sharing here: while it is often tempting to skip stage 2 (participant reaction) as it does not necessarily correlate with stage 3 (job impact), enabling your learners to feed back on their experience not only provides valuable information to enable you to improve future training offerings, but makes learners feel as though their voices and engagement in the training matter, which can assist with maintaining motivation for ongoing professional development. Furthermore, remembering that your learners are human, need to feel like they belong, and their company cares about them is a key part of sustaining engagement. Learner participation in future training is likely to improve if you take into account their feedback and respond by making changes to enhance the experience – few people wish to sit through tedious or unpleasant training experiences, whether or not it ultimately improves their performance!