Skip to content
PMScore Logo
BlogKnowledge

Learning management system (LMS): what it does and why it does not prove learning impact

Which features a learning management system offers, what to look for when choosing one and which question an LMS alone does not answer: whether training works.

In many companies, a learning management system (LMS) is the backbone of training. It manages courses, distributes e-learning, sends reminders about mandatory training and records who completed what. Anyone choosing an LMS today compares features, integrations and prices.

One question often gets lost: how will you know later whether the trainings in the LMS have any effect? This post explains what an LMS does, what to look for when choosing one and where its limits are.

What is a learning management system?

A learning management system is software that organizations use to provide, assign, deliver and record learning. Typical features are:

  • Course management: classroom trainings, webinars and e-learning in one catalog, with dates and registration
  • Assignment: mandatory training by role, department or location, with reminders and deadlines
  • E-learning: playing learning content, usually in the SCORM or xAPI standard
  • Tests and certificates: knowledge checks, proof of completion and certificate expiry dates
  • Reports: completion rates, learning time, completed mandatory training

Well-known systems for larger companies include SAP SuccessFactors Learning, Cornerstone and Workday Learning, and for educational institutions and smaller organizations, for example, Moodle. A comparison with a view to evaluation is in the post Alternatives to SAP SuccessFactors Learning.

What an LMS shows and what it does not

An LMS knows very precisely what happens in the system. What happens afterwards at work lies outside its data:

  • What an LMS shows

    Who attended and completed which course, how long it took, who passed the test and how satisfied participants were.

  • What an LMS does not show

    Whether participants feel able to apply it, whether they intend to, whether they do, and why it does not work when it does not.

Many LMS do offer feedback forms and later follow-up evaluations. Which questions they contain, however, is up to you. Often they ask about satisfaction, which is not related to later behavior at work (Alliger & Janak, 1989; Alliger et al., 1997). A passed test shows knowledge, but not yet application. Why that is so is explained in the post Assessing learning outcomes in training.

On top of that there is a practical gap: trainings by external providers often do not run through the client's LMS at all. They appear in no report.

What to look for when choosing an LMS

Besides features, usability and cost, four questions about evaluation are worth asking:

  1. Which metrics does the system provide out of the box? Almost every LMS provides completion rates and learning time. Ask which metric shows whether trainings work.
  2. Who writes the evaluation questions? If you have to develop them yourself, plan time and expertise for it.
  3. Are results comparable across courses? Only with the same questions in every course can you compare formats, providers and versions.
  4. How do you capture trainings outside the system? External seminars and classroom trainings need a measurement that works without access to the LMS.

If the LMS does not answer these questions, that is no reason to rule it out. It only means you have to solve impact measurement alongside it.

Combining LMS and transfer measurement

PMScore does not replace an LMS. It runs alongside it and provides the metric the LMS lacks: the Transfer Effectiveness Score (TES) from 0 to 100. At the end of the course, participants answer a short, anonymous assessment on competence gain, ability to apply and transfer intention, via link or QR code, without an account. This works for courses in the LMS just as well as for external seminars.

Because the questions are the same in every course, you also see whether a revision worked:

The values are made up. In the example, the first version taught a lot but built little confidence to apply it. After a revision with more practice cases, the ability to apply rises most. In the LMS both versions would have looked equally successful: same completion rate, test passed.

The TES is based on participants' self assessment and does not replace observation at work. But it makes transfer effectiveness measurable in every course with little effort. Which metrics belong in a dashboard alongside it is explained in the post Learning analytics in learning and development.

What to take away

  1. Choose an LMS for what it does well: managing, distributing and recording learning.
  2. When choosing, settle how you will measure the impact of trainings, in the system or alongside it.
  3. Measure transfer effectiveness in every course with the same questions, including where the LMS does not reach.

Add impact measurement to your LMS?

Tell us which LMS you use or are currently choosing. In a short conversation we show you how the TES runs alongside it.

Thanks! We'll get back to you soon.

That didn't work right now. Please try again later or email us directly.

Sources

  • Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick's levels of training criteria: Thirty years later. Personnel Psychology, 42(2), 331-342.
  • Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341-358.

SAP SuccessFactors, Cornerstone, Workday and Moodle are trademarks of their respective owners. This post is independent and not affiliated with these companies.