Skip to content
PMScore Logo
BlogKnowledge

Analyzing a Likert scale: methods, pitfalls and an example from training evaluation

How to analyze answers on a Likert scale properly, when an average is justified and how individual agreement ratings become a comparable metric.

"Strongly disagree" to "strongly agree": hardly any questionnaire gets by without a Likert scale, including the feedback form after a seminar. The answers are quickly collected. The harder question is what you may calculate with them and what they actually say.

This post shows the common methods, works through an example from training evaluation and names the pitfalls that analyses most often stumble over in practice.

What is a Likert scale?

The Likert scale goes back to the social scientist Rensis Likert (Likert, 1932). Respondents rate statements by how much they agree with them, in fixed steps, for example from 1 (does not apply at all) to 7 (fully applies).

Strictly speaking, a single such statement is a Likert item. The Likert scale is the combination of several items that measure the same attribute, for example three statements on the ability to apply what was learned after a training. This distinction matters for the analysis, as the next section shows.

How many points? Five and seven are the most common. In a comparison study, scales with seven to ten points performed better on reliability and discriminating power than scales with few points, without respondents finding them more tedious (Preston & Colman, 2000).

May you calculate an average?

This is the best known debate. Strictly speaking, Likert answers are ordinal: the points have an order, but nobody guarantees that the distance between 2 and 3 is the same as between 6 and 7. For ordinal data, the median and frequencies are the appropriate statistics (Jamieson, 2004).

The opposing view: means and common statistical methods yield robust results with Likert data too, especially for scales made of several items (Norman, 2010). In practice, a simple rule has proven itself:

  • Single item: show the distribution, plus the median or agreement rate.
  • Scale of several items: the mean of the items is justifiable and easy to compare.

If you always keep an eye on the distribution as well, you will not miss a split group behind an unremarkable average.

The key methods of analysis

  • Frequency distribution: how many chose which point? The most honest representation, as bars or a table.
  • Agreement rate: share of answers in the agreeing range, on a 7 point scale for example points 5 to 7. Easy to understand and good for reports.
  • Top box and top 2 box: share of the highest or the two highest points. Shows how many are truly convinced.
  • Median: the middle value when all answers are sorted by size. Insensitive to outliers.
  • Mean and standard deviation: for scales of several items. The standard deviation shows how much the group agrees.
  • Conversion to 0 to 100: (value minus 1) divided by (points minus 1), times 100. On a 7 point scale, 1 becomes 0, 4 becomes 50 and 7 becomes 100. That way results from different scales can be put side by side and reported clearly.

An example from training evaluation

50 participants of a training rate the statement "I feel able to apply what I learned in my own work." on a scale from 1 (does not apply at all) to 7 (fully applies). The numbers are made up. This is how the answers are distributed:

Answers per point, 50 participants

  • Point 1: does not apply at all (1 answer)2 %
  • Point 2 (2 answers)4 %
  • Point 3 (3 answers)6 %
  • Point 4 (6 answers)12 %
  • Point 5 (12 answers)24 %
  • Point 6 (16 answers)32 %
  • Point 7: fully applies (10 answers)20 %
Made up example values. The median is highlighted.

This gives:

  • Agreement rate (5 to 7): 38 of 50, so 76 percent.
  • Top 2 box (6 and 7): 26 of 50, so 52 percent.
  • Median: 6, because the 25th and 26th answers are both on point 6.
  • Mean: 264 divided by 50 gives 5.28.
  • On 0 to 100: (5.28 minus 1) divided by 6, times 100 gives about 71.

Each number tells a different part of the story. The agreement rate sounds good, the top 2 box shows that only half are truly convinced. And twelve percent are below the midpoint: for them, it is worth asking why.

Common pitfalls

  • The midpoint is not "don't know". People without an opinion often choose the midpoint and so distort the result. If needed, offer a separate answer "cannot judge" and do not count it.
  • Acquiescence. Many people tend to agree with statements rather than not. Phrase statements concretely and avoid leading questions.
  • Mixed polarity. Negatively worded items must be reverse coded before combining, otherwise the values cancel each other out.
  • Compare only on the same scale. A 4.2 on a 5 point scale is something different from a 4.2 on a 7 point scale. Convert to 0 to 100 for comparisons.
  • Small groups. With eight answers, a single person shifts the agreement rate by more than twelve percentage points. Always report the number of answers.

From a single item to a comparable metric

In training evaluation, a single item is rarely enough. It becomes meaningful when several items are combined into components, converted to 0 to 100 and formed the same way in every course. That is exactly how PMScore works: at the end of the course, participants anonymously answer statements on a 7 point scale, and PMScore combines them into three components:

  1. Competence gain

    How much have participants learned?

  2. Ability to apply

    Do they feel able to use what they learned in their own work?

  3. Transfer intention

    Do they actually intend to put it into practice?

Together they make up the Transfer Effectiveness Score (TES) from 0 to 100, which makes courses, formats and providers comparable. The distribution of each individual statement stays visible in the report, so that a split group does not disappear in the average.

Which statements suit the form after a seminar is shown in the post Feedback forms for seminars and training.

What to take away

  1. For single items, show the distribution plus the median or agreement rate.
  2. Calculate means mainly for scales of several items.
  3. Convert to 0 to 100 for comparisons and reports.
  4. Always state the number of answers and look specifically at the group below the midpoint.

Stop analyzing your assessments by hand?

Tell us how you survey and evaluate your trainings today. In a short conversation we show you how PMScore turns the answers into the TES and a comparable report automatically.

Thanks! We'll get back to you soon.

That didn't work right now. Please try again later or email us directly.

Sources

  • Jamieson, S. (2004). Likert scales: How to (ab)use them. Medical Education, 38(12), 1217-1218.
  • Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1-55.
  • Norman, G. (2010). Likert scales, levels of measurement and the "laws" of statistics. Advances in Health Sciences Education, 15(5), 625-632.
  • Preston, C. C., & Colman, A. M. (2000). Optimal number of response categories in rating scales: Reliability, validity, discriminating power, and respondent preferences. Acta Psychologica, 104(1), 1-15.