Skip to content
PMScore Logo
BlogKnowledge

SAP SuccessFactors Learning: what it does for training evaluation and where the gap lies

Which levels of training evaluation SAP SuccessFactors Learning covers, why transfer often remains unanswered and how to measure it next to your LMS.

In many organizations, SAP SuccessFactors Learning is the learning management system (LMS) for learning and development. It schedules courses, assigns mandatory training, and tracks completions and certificates.

Training evaluation in SAP SuccessFactors Learning

SAP bases evaluation in SuccessFactors Learning on Kirkpatrick's four level model (Kirkpatrick, 1959). According to the SAP Learning Academy, level 1 and level 3 are supported through surveys and level 2 through assessments (SAP, n.d.). There are two survey types: course feedback on satisfaction with the course and the follow-up survey, which is assigned automatically after a set period. SAP does not mention level 4 there.

After the training

  1. Level 4ResultsNot named as supportedWhat does the training bring the organization?
  2. Level 3BehaviorThis is where the value is decidedDo participants apply what they learned? In SuccessFactors through a follow-up survey after a set period.

In the training

  1. Level 2LearningWhat have participants learned? In SuccessFactors through assessments.
  2. Level 1ReactionWhere most companies measureHow did participants like the training? In SuccessFactors through course feedback.

So the tools are there. How meaningful the evaluation becomes depends on which questions you ask and when. The post The Kirkpatrick model: training evaluation in four levels explains the levels in detail. What tests on level 2 show and where they stop is covered in the post Assessing learning outcomes in training.

Where the gap lies

An LMS is built to organize learning. Measuring transfer is a toolkit there, not a ready metric. In practice, three gaps show up:

  • You define the questions yourself. You build the surveys in SuccessFactors yourself. They often end up with the questions of the classic feedback form: how satisfied were you with the trainer, the content and the organization? Two meta analyses show that satisfaction is not related to later behavior at work (Alliger & Janak, 1989; Alliger et al., 1997). A metric that makes courses, formats and providers comparable is something you first have to develop yourself.
  • Level 3 comes late. A follow-up survey measures after weeks or months. By then the course has often run several times, and the results hardly help the next run.
  • External trainings stay outside. Many trainings are run by external trainers and training providers. They usually have no access to their clients' LMS and can neither measure nor prove their effectiveness there.

Many L&D teams know the result: the LMS knows exactly who completed which course. Whether the course had any effect is not in it.

Using SuccessFactors Learning and PMScore together

PMScore does not replace an LMS. Courses, assignments and certificates stay in SuccessFactors. PMScore adds the metric that is missing there: the Transfer Effectiveness Score (TES). At the end of the course, participants answer a short, anonymous assessment, via link or QR code, without an account and without access to the LMS. It does not ask about satisfaction, but about what matters for transfer:

  1. Competence gain

    How much have participants learned?

  2. Ability to apply

    Do they feel able to use what they learned in their own work?

  3. Transfer intention

    Do they actually intend to put it into practice?

Together they make up the TES from 0 to 100. The values in the example above are made up. Because every assessment yields the same TES, you compare courses, formats and providers directly, including trainings by external providers that never appear in the LMS. Where the value drops, PMScore asks why. The result is there at the end of the course, not months later.

The TES is based on participants' self assessment. It does not replace observation at work, but it makes transfer effectiveness measurable in every course with little effort. A follow-up survey in SuccessFactors remains a useful addition for high budget trainings. The post Training controlling: steering learning and development with metrics that show transfer shows how to build both into your reporting.

What to take away

  1. Use SuccessFactors Learning for what it is built for: managing, assigning and recording courses.
  2. Check which questions your surveys ask today. Questions about satisfaction say little about transfer.
  3. Add a metric for transfer effectiveness that is available at the end of the course, the same in every course, and covers external trainings too.

Considering a switch anyway? The post Alternatives to SAP SuccessFactors Learning gives an overview. What an LMS can do in general is shown in the post Learning management system (LMS): what it does and why it does not prove learning impact.

Use the TES next to SuccessFactors?

Tell us which trainings you manage today in SuccessFactors or another LMS. In a short conversation we show you how the TES runs alongside it and what you can report from it.

Thanks! We'll get back to you soon.

That didn't work right now. Please try again later or email us directly.

Sources

  • Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick's levels of training criteria: Thirty years later. Personnel Psychology, 42(2), 331-342.
  • Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341-358.
  • Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training Directors, 13(11), 3-9.
  • SAP (n.d.). Introducing Training Surveys. SAP SuccessFactors Learning Academy. learning.sap.com

SAP and SAP SuccessFactors are trademarks of SAP SE. This post is independent and not affiliated with SAP.