I work for a market research company doing surveys over the phone, and one of the common sort of questions that gets asked in these surveys is the "On a scale of one to ten, where one is strongly disagree and ten is strongly agree..." and I've noticed something about this sort of question: It's complete bullshit. No one answers it properly.
Five is not the Middle
When many people are asked this sort of question and they have no real feelings one way or another, they say five. Makes sense, five is half of ten, so it must be a neutral score, right? Well, no. There are an even number of possible scores. There is no neutral. Five is slightly disagree (or slightly dissatisfied or whatever the actual scale is).
Ten is Too High
Another weird thing people do when answering this question is refuse to use the full scale. When asked, "on a scale of one to ten where one is extremely dissatisfied and ten is extremely satisfied, how satisfied are you with X?" a lot of people will give a score of eight. Nothing wrong with that, but then when you ask them "And what could be improved to increase your satisfaction?" they'll tell you "Nothing, it was fine, I just don't give ten out of ten."
This undermines the entire purpose of the scale. These people, when asked to give a score from one to ten, actually give a score from one to eight. Even though there is absolutely nothing they can think of that would have improved the experience they're being quizzed on, they still will not give a score that reflects that. They won't even give a nine, apparently.
One is Too Good
People seem to have no such compunctions about giving one out of ten though. In fact, people who were unhappy about the experience will often try to give zero, even though that's not even on the scale. Along with the lack of a neutral value, this makes the whole thing lean fairly heavily toward the negative.
Too Many Numbers
Another problem is that people just can't break things down into ten distinct levels of agreement/satisfaction/whatever. They think they can, and in fact will try to break it down further (if the software we use allowed it, there'd be a lot of 8.5s and 9.5s), but no one really uses the whole scale. Even the people who actually listen to the questions and try to answer each one properly will use the same few scores for all of them.
Ask twenty questions and even if you've got someone who won't just say "Eight" or "Ten" or "One" to every single question, regardless of what you're actually asking about, you'll still get a mix of between three and six different scores.
How to Fix It
Not all the surveys I've worked on use this scale, and there are better alternatives. Some, for example, use zero to ten, which is an improvement, because five actually is a neutral score there, as people expect it to be, but it doesn't solve all the problems. People still have this strange reticence about giving ten out of ten and there are still too many possible answers.
A scale of one to seven seems to work better. There's a neutral score (four), the highest score is low enough that people don't seem to avoid giving it out, and the fact that people aren't expecting a scale of one to seven seems to make people actually think about it a bit more, but there are still more possible answers than people are likely to use.
The Best Solution
The best scale, as far as I can tell, is one to five. It's got a neutral value, no one's afraid to give the highest possible score, and the five available options map directly to some pretty obvious answers. Strongly negative, moderately negative, neutral, moderately positive, strongly positive.
People understand this intuitively and will use the full range. So if you're looking to get people to rate something, this is the scale to use.
No comments:
Post a Comment