When the question changes the answer
The most-used rating scale in exercise science measures different things in different studies, and small choices in how the question is asked explain why.
Published: Sports Medicine, 2020 (co-author)
Type: Methodological review
Reach: 160+ citations
The problem
Ratings of perceived effort are the most frequently used single-item scales in exercise science, much like the ease or satisfaction ratings product teams rely on. Yet studies define effort differently, put different terms in the question, use different versions of the scale, and give different instructions.
What varies between studies:
Definition: what “effort” means
Terms: what the question names
Scale: e.g., 6–20 or 0–10
Instructions: what people are told to rate
What we did
We mapped those choices and argued that the way they combine threatens whether the scale measures what researchers think it measures. We recommended narrowing the choices and adding separate single-item scales for feelings, fatigue and discomfort, so one question is not asked to carry several meanings.
What it shows
Before any survey goes out, mine or a stakeholder’s, I check the same things: will every respondent read the item the same way, do the anchors and instructions match what we mean to measure, and is one question doing the work of several?
Read the paper: Rating of perceived effort: Methodological concerns and future directions (Sports Medicine, 2020).
