Blended Evaluation: Moving from the Past to the Future
Last week, we discussed possible sources of data and methods for gathering it when embarking upon blended evaluation. This week, we continue our conversation about hybrid approaches by busting one of the most common beliefs in the world of evaluation and about the Kirkpatrick Model.
The extremely counterproductive belief that has been standard practice for decades is that each Kirkpatrick level should be assessed individually. This results in learning and development (L&D) professionals distributing post-course surveys, commonly called “evals,” “smile sheets” or “Level 1s,” which only address issues of immediate participant reaction to the training and the trainer.
At the end of a program, we often see “Level 2s,” which most commonly take the form of knowledge tests or skills tests. Again, there is little or no connection to any other levels.
“Level 3s,” if they occur at all, typically take the form of a survey sent to training grads and their supervisors 60 or 90 days following the completion of a formal program. While it is good that the bridge between training and the business is being crossed in these cases, evaluation at this level often amounts to little more than asking, “How are things going over there?”
Finally, “Level 4s” are often non-existent because those without a working knowledge of the Kirkpatrick Model are unsure of how to assess Level 4.
The bedrock of the Kirkpatrick Model is that the levels are best assessed collectively, not separately. The definition of blended evaluation in our world is, “Tools that measure multiple levels at the same time, such as a survey with questions related to both Levels 1 and 2 immediately following training.”
One example of hybrid or blended evaluation is an instructor observing a small group during classroom activity, looking for each individual participant’s level of involvement (Level 1) and ability to demonstrate certain skills (Level 2). At the end of the activity, the instructor “pulls up a chair” and leads a discussion on how they might apply what they just did on the job (Level 3) and how their customers will likely respond to their efforts (Level 4).
Another example would include the following focus group questions targeted to a group of supervisors in regards to training their employees completed two months prior:
1. What have you seen your employees doing differently over the past 2 months? (Level 3)
2. To what can you attribute their improvement? (Levels 2 and 3)
3. What signs of success (Level 4) have you seen from their efforts (Level 3).
A final example of blended evaluation can be found within the Kirkpatrick blended evaluation tool samples available to you for free.
Additional Resources
Kirkpatrick Four Levels® Evaluation Certification Program – Bronze Level
