By CustomerSat Research & Consulting Services

What scale should you use in your survey’s rating questions? There are many options (e.g., an eleven-point scale, a ten-point scale, a seven-point scale, or a five-point scale), each with its pros and cons. In fact, at least 1,536 different rating scales could be chosen.

Basically, there are seven factors to consider in your decision:

1. Type of scale Two choices: unipolar/bipolar
2. Scale balance Two choices: balanced/unbalanced
3. Midpoint Two choices: presence/absence of midpoint
4. Inner labeling Four choices: none/numeric/non-numeric/midpoint only
5. Provision of Not Applicable Two choices: presence/absence of Not Applicable option
6. Response Requirement Two choices: mandatory/optional
7. Number of scale points Twelve choices: 2 through 12, infinite.

Yet even these seven factors do not cover all possibilities. There could be more factors (e.g., presentation in vertical/horizontal format). And, there could be more choices for the specified factors (e.g., a twelve-point scale).

Type of Scale

A scale can be either bipolar or unipolar, meaning it either reflects two contrasting extremes or it reflects absence/presence of a construct, as seen in these examples:

Scale Balance and Midpoint

Unipolar scales are by definition unbalanced or asymmetrical. Bipolar scales can either be balanced or unbalanced, and with or without midpoint. Consider these examples:

Excluding a midpoint (e.g., Neither satisfied nor dissatisfied) forces respondents to give non-neutral responses. Including a midpoint can be an advantage because some respondents might truly be neutral. If they are not offered the choice of a neutral response, respondents might skip the question or give a less-than-accurate answer, e.g., more positive than they mean.

Inner Labeling

Should all response choices be labeled? If so, should they bear descriptive labels, numeric ones or both? Or, should only the midpoint be labeled? Labeling every response choice gives the respondent more guidance and can make it easier to communicate the asymmetrical nature of an unbalanced scale. However, there is a downside of labeling every response. For example, the "distance" the respondent perceives between "Neither dissatisfied nor satisfied" and "Somewhat satisfied" may not be the same as the distance between "Somewhat satisfied" and "Completely satisfied." In addition, labels may have different meanings in different cultures/countries. It can be hard to consistently translate such subtle distinctions as "somewhat" and "slightly" into certain languages.

Not Applicable (NA), No Opinion, Don't Know Choices

What if a question is not applicable to a respondent and the respondent does not have the choice of answering NA? The respondent will either leave the response blank (if that is an option) or give an answer that may not accurately reflect their actual opinion. For example, the person may respond with a neutral or near-neutral response if the question does not apply in their case. As a result, the person is lumped with someone who does have an applicable experience but chooses a neutral response.

If respondents face the decision of answering questions that do not apply to them, it can increase respondent fatigue and/or frustration. This could possibly affect the quality of the responses and whether the respondents will complete the survey.

Response Requirement

Should you require a response to a particular question? This question is particularly relevant when a respondent can be prevented from proceeding unless the question is answered, e.g., surveys taken over the Internet, by telephone, and in-person interviews.

There are four cases of this question. First, the respondent can be required to answer the question but not given an NA choice. Being forced to give an answer may yield more data, but the data might not be as accurate. The respondent might also become frustrated and abandon the survey. That would result in less data and, possibly, annoyance toward the surveying organization.

A second case would also force the respondent to answer the question, but provides an NA option. Having the NA option means less frustration for the respondent, but increases the NA selections.

The remaining two cases involve not requiring a response, either with or without an NA option. These cases can be less taxing on the respondent. The disadvantage is that the respondent may fall into a pattern of skipping responses.

Number of Scale Points

The optimal number of scale points is a matter of some debate. A wide range of opinions, many of them contradictory, have appeared in scholarly articles on the subject. For example:

"Taken as a whole…the optimal length of a rating scale is 5 to 7 points."

"The debate in the literature is ongoing; however, it is safe to say that 4- or 5-point scales will be serviceable for most attitude or opinion data collection."

"Overall, the data suggest that the 10-point scale results in somewhat more dispersion and lesser positive skew than the 7-point scale."

"[W]hile our basic recommendation is to use 5-point scales, we are comfortable using 10-point scales for some surveys, particularly those conducted via telephone."

"In general, there is no single, optimal number… for all scaling situations."

Fred Reichheld states the situation well. "Talk to ten research firms, and you will hear ten different arguments for the best scale to use… yes/no, three choices, four, seven, whatever; each one advocated with near-religious fervor." He goes on to state examples of a successful 3-point scale, a successful 5-point scale, and the advantages of his preferred scale, 0-10.

To further complicate the issue, some Web surveys may now allow respondents to mark a point along a line that is labeled only at the endpoints. Survey software then calculates a score for it, e.g., 5.6 on a 1-10 scale.

CustomerSat Recommends…

So where do all these options leave us? CustomerSat has found the following factor sets helpful for rating questions like overall satisfaction.

Type of Scale We usually use a bipolar scale. The rationale is that there isn’t merely the absence or presence of satisfaction — levels of satisfaction anchored on the low end with no satisfaction must be included. Individuals can be dissatisfied "lower" than no satisfaction.

Scale of Balance and Midpoint CustomerSat uses a balanced (symmetric) scale with or without a midpoint. The satisfaction rating scale is generally: 1–Completely or Very Dissatisfied through 10–Completely or Very Satisfied. Sometimes 5 is specified as a midpoint, even though it’s not truly the middle of a 1-10 scale (5.5 is the middle).

Inner Labeling With the possible exception of the midpoint, CustomerSat does not typically use inner labeling other than the numbers 1–10. There are several reasons but the major one, mentioned previously, is that text labels sometimes have different meanings in different countries and cultures.

Not Applicable (NA) Option Generally, CustomerSat supports the use of Not Applicable responses. The principal reason is that there may truly be situations where the question does not apply to the respondent (e.g., asking their overall satisfaction with financing when the respondent paid cash). Another reason is reducing respondent fatigue. Respondents shouldn't have to agonize over questions that don't apply to them. Nor do we want them getting into a habit of not answering questions. If offering NA truly does not make sense (e.g., the question applies to every respondent), then the NA option may be omitted.

Response Requirement To avoid frustration, CustomerSat does not usually require respondents to answer a rating question. Frustration can lead to abandoning the survey before it’s completed, and might reflect poorly on the surveying organization. Sometimes, however, the overall satisfaction (OSAT) response is required. It is only one question of many, but it’s very important to the analysis, and should apply to every respondent.

Number Scale CustomerSat generally employs a 1–10 scale for rating questions. Virtually everyone has ten fingers and operates in a base-ten world. In addition, the 10-point scale can possibly increase sensitivity and discrimination, clarity and comprehensibility, and suitability for correlation and regression. This also provides standardization for our benchmark data. Therefore, one requirement for most of the questions in our benchmark program is use of a 10-point scale. (Note that the recommended scale for IVR [Integrated Voice Response] surveys is 0–9 because of the limitation of digits available on a telephone touchpad.)

To summarize, CustomerSat generally uses:

• Bipolar scaling
• Balanced scales
• No midpoint
• Numeric inner labeling
• NA response options where applicable
• Not requiring an answer
• 1-10 scale

However, one size rarely fits all, and certain situations dictate different choices. Fortunately, the CustomerSat Enterprise™ solution is flexible enough to handle virtually any combination of choices. The optimal solution, of course, is to tailor the options to your individual situation. Our Research & Consulting Services team is familiar with the pros and cons of all seven factors, and can customize your choices to provide the mix that best fits your individual situation.

For details, and to discuss further, please contact your CustomerSat representative, email us at expert@CustomerSat.com, or call us at 650.237.3300.

Selected References

Chrzan, Keith and Joey Michaud, "Response Scales for Customer Satisfaction Research," Quirk's Marketing Science Review (October 2004).

Feltser, E.B., "The Numbers Game: Refining Multi-Point Scales," Quirk's Marketing Research Review (March 1997).

Krosnick, Jon A. and Leandre R. Fabrigar, "Designing Rating Scales for Effective Measurement in Surveys," in Lars Lyberg, et al, editors, Survey Measurement and Process Quality (New York, New York: John Wiley & Sons, Inc., 1997), pp. 141-164.

Lewis, Steven, "The Language of International Research: 'Very Satisfied' and 'Totally Satisfied' are Not The Same Thing," Quirk's Marketing Research Review (November 1997).

Peterson, Robert A., Constructing Effective Questionnaires (Thousand Oaks, California: Sage Publications, Inc., 2000).

Quirk's Staff, "Rating Scales can Influence Results," Quirk's Marketing Research Review (October 1986).

Reichheld, Fred, The Ultimate Question: Driving Good Profits and True Growth (Boston, Massachusetts: Harvard Business School Press, 2006).

Sudman, Seymour and Norman M. Bradburn, Asking Questions (San Francisco, California: Jossey-Bass Publishers, 1982).

Sue, Valerie M. and Lois A. Ritter, Conducting Online Surveys (Los Angeles, California: Sage Publications, Inc., 2007).

Waddell, Howard, "A Better Customer Satisfaction Scale," Quirk's Marketing Research Review (October 1999).