Skip to content
PMScore Logo
BlogKnowledge

The Kirkpatrick model: training evaluation in four levels

What the four Kirkpatrick levels measure, why learning transfer on level 3 is the hard one, and how you measure learning transfer instead of asking only about satisfaction.

Almost every company evaluates its trainings. According to ASTD Research (2009), 92 percent of companies use feedback forms, usually on the last day of the training. What they measure is mostly satisfaction: with the trainer, the materials, the hotel. Whether what was learned arrives in everyday work afterwards, these forms do not show.

The best known model of training evaluation makes this gap visible: the four level model by Donald Kirkpatrick. This post explains the four levels, shows why the third level in particular is so hard to capture, and how you can measure learning transfer.

The four levels of the Kirkpatrick model

Donald Kirkpatrick described his model in a series of articles at the end of the 1950s (Kirkpatrick, 1959). To this day it is the basis of almost every Kirkpatrick evaluation in learning and development. It sorts the success of a training into four levels:

Field of application: at work

  1. Level 4ResultsWhat does the change in behavior bring the organization? For example fewer errors, shorter lead times or happier customers.
  2. Level 3BehaviorThis is where transfer happensDo participants apply what they learned at work? Only when their behavior changes or widens do we speak of transfer.

Learning field: in the training

  1. Level 2LearningWhat knowledge and skills have participants gained?
  2. Level 1ReactionWhere most companies evaluateHow satisfied are participants with the training?

The two lower levels concern the training itself, the learning field. The two upper levels concern the workplace, the field of application (Weinbauer-Heidel, 2016). Most companies evaluate only on level 1, exactly where the least is decided about the success of a training.

Why level 3 is the hard one

For a long time it was assumed that the levels build on each other: whoever is satisfied learns more, applies more and achieves better results. A good feedback form would then at least be an indirect measure of transfer. Two meta analyses refuted this assumption (Alliger & Janak, 1989; Alliger et al., 1997). Satisfaction with a training says nothing about whether what was learned is applied later.

There is a simple reason for this. At the end of a training, participants usually do not know themselves how much they will put into practice. Transfer happens weeks later at work, out of the trainers' reach. Observing it directly takes time, needs managers as observers and often reaches only part of the participants.

Measuring only level 1 therefore means flying blind. Yet trainings follow the same rule as every other metric: what gets measured gets done (Burke & Hutchins, 2008). What is not measured does not get improved.

A first lever lies in how you ask. The 1997 meta analysis separates satisfaction from usefulness. Questions about satisfaction show no significant relation to later behavior at work. Questions about how useful the training is for one's own work correlate with transfer, if only weakly (r = 0.18). What seems useful is more likely to be used.

Measure learning transfer with PMScore

PMScore starts right there. Instead of asking if the training was enjoyable, it asks what matters for transfer. At the end of the course, participants answer a short, anonymous assessment. This generates three values:

  1. Competence gain

    How much have participants learned? This corresponds to Kirkpatrick's level 2.

  2. Ability to apply

    Do they feel able to use what they learned in their own work? This is the bridge from the learning field to the field of application.

  3. Transfer intention

    Do they actually intend to put it into practice? Those who plan nothing rarely implement anything.

Together they make up the Transfer Effectiveness Score (TES) from 0 to 100. Ability to apply and transfer intention do not yet observe behavior, that only happens at work. But they ask about use and application instead of satisfaction, the kind of reaction that research links to later transfer.

Because every assessment yields the same TES, courses, formats and versions of a training can be compared with each other. Where the value drops, PMScore asks why. So you see not only whether transfer succeeds, but also where to start improving.

How the follow-up questions work is explained in the post New in PMScore: follow-up questions and the assessment after the exam.

What to take away for your training evaluation

  1. Ask about usefulness and application instead of satisfaction.
  2. Measure competence gain, ability to apply and transfer intention at the end of every course, always with the same questions.
  3. Compare the values across courses and versions and improve where transfer drops.

Want to use PMScore?

Let’s show you in a quick call what PMScore reveals about your trainings.

Sources

  • Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick's levels of training criteria: Thirty years later. Personnel Psychology, 42(2), 331-342.
  • Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341-358.
  • ASTD Research (2009). The Value of Evaluation. ASTD Press.
  • Burke, L. A., & Hutchins, H. M. (2008). A study of best practices in training transfer and proposed model of transfer. Human Resource Development Quarterly, 19(2), 107-128.
  • Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training Directors, 13(11), 3-9.
  • Weinbauer-Heidel, I. (2016). Was Trainings wirklich wirksam macht. 12 Stellhebel der Transferwirksamkeit.