|
|
In
this Issue of Connections
A Monthly
Newsletter from CustomerSat
Top Box, Bottom Box, Mean
Score - What's Important?
By CustomerSat.com
Client ServicesTop
box, bottom box, and mean score are different ways of evaluating
performance of a rated attribute. “Top one box” is the percentage of customers who give the
attribute the highest rating, e.g., 10 on a 10-point scale or 5 on a
5-point scale. “Top two
box” is the percentage who give the attribute either the highest
rating or second highest rating on the scale; similarly for “top
three” box. For 5-point scales, top-one and top-two box percentages are most often used; for
10-point scales, top-two and top-three (rating scores of 8, 9, or 10)
box percentages are more often used. (In this article, we will use “top box” to refer to any of
top one, top two, or top three box.) “Bottom box” percentages are the same for the lowest ratings
of the scale.
In
contrast to top and bottom boxes, mean scores are not percentages but
the average of all customers’ ratings of an attribute.
Should you use top or bottom boxes or mean scores?
Or all three?
No
Single Statistic tells the Whole Story
Top
and bottom box percentages reflect the responses of only those customers
who give the top or bottom box ratings. So it is quite possible for top or bottom box percentages to rise
or fall while mean scores – which reflects the experiences of all
customers – move in the opposite direction. Clearly, top or bottom
boxes alone do not tell the entire story.
But
neither do mean scores alone. With
a mean score of, say, 7 on a 10-point scale, where 10 is outstanding,
the percentage of customers rating performance as outstanding (top one
box) can vary anywhere from 0% to 70%, and the percentage of customers
rating performance as unacceptable (bottom one box) can vary anywhere
from 0% to 30%. Mean
scores tell us how we are doing overall, but they rarely tell us whether
we are providing truly exceptional service experiences or ensuring
against unacceptable ones.
To
avoid these hazards, performance measurement systems should generally
use a combination of mean scores and one or both of top box or bottom
box percentages. Some very
comprehensive systems have specific targets for all three: for example,
mean score of 8.0 out of 10 or better; top-3 box of 80% or higher;
bottom-3 box of 1% or less. Usually, however, it is sufficient to use mean score and either top box or bottom
box.
Attribute’s
Correlation with Overall Outcomes Helps Prioritize Top vs. Bottom Box
Whether
we use top box or bottom box depends upon how the attribute is
correlated with the key overall outcome (dependent) variable, be it
overall satisfaction, retention, advocacy, or more generally,
“loyalty”. In simple
regression analysis, we assume a linear or “neutral” relationship
between performance of the attribute and loyalty, as shown in Figure 1. If the relationship is indeed linear, we can simply focus on
maximizing mean scores, without regard to top or bottom box percentages.
Figure 1: Regression Analysis
But
a technique called Non-Linear Effect Analysis shows us that in most
cases, loyalty rises not linearly, but exponentially with performance of
the attribute. Customers rating their experiences a 10 are usually much more
loyal relative to those rating their experiences an 8, than are, say,
customers rating their experiences an 8 relative to those rating their
experiences a 6. In these cases, customers rating their experiences in
top box categories will contribute far more than their fair share to an
organization’s long-term revenues and profits. We call these “satisfaction enhancing” attributes, for which the
greatest payoff is to maintain outstanding levels
of performance. For these attributes, it is most important to
monitor, and take actions to raise top box percentages.
Figure 2: Non-Linear Analysis
For
other attributes, customer expectations may be low. Retention and
advocacy rise most steeply with increasing satisfaction on the lower
half of the scale. In these
cases, customers rating their experiences a 5 are much more loyal
relative to those rating their experiences a 3, than are, say, customers
rating their experiences an 10 relative to those rating their
experiences a 8. We call
these “satisfaction maintaining” attributes, for which the greatest
payoff is to maintain moderate levels of performance. For
these attributes, it is most important to monitor, and take actions to reduce
bottom box percentages.
If
the appropriate combination of mean scores, top box and/or bottom box
are not included in customer satisfaction management systems, investment
decisions maybe sub-optimal. For
these reasons, CustomerSat’s online reporting always includes mean
scores and the top box and bottom box percentages appropriate to a
particular attribute’s rating scale, as demonstrated in this example
(click image for a larger picture):
Figure 3. Rating Summary Scores are powerful tools for
market research professionals
If you are interested in exploring refinements
to your customer satisfaction measurement and reporting metrics, please
visit www.CustomerSat.com
or contact a CustomerSat.com representative at (800) 372-7772.
Eliminate Biases with
Automated Randomization and Random Selections
A key to successful
research is ensuring that respondents are not biased by factors
irrelevant to the research, such as the order in which a question’s
choices are listed or the order of questions in the questionnaire. Sometimes, respondents will select the first choice in a list,
regardless of the question. Or
respondents may spend more time with questions near the beginning of a
questionnaire than those later. Rotations, randomizations, and random
selections address these biases.
Rotations shift
items (either choices or questions) through a fixed order, starting with
the first item for the first respondent, the second item for the second
respondent, and so on. If
there are N items, every Nth respondent is presented with the items in
the same way. In contrast,
randomizations present the items in no fixed order, but present each of
N items with probability of 1/N in any particular position.
From a research standpoint, randomizations are generally superior
to rotations because they randomize not just the first item in the list,
but also all other items in the list.
Random selections are
used when the number of items exceeds what we can reasonably ask
respondents to consider, such as which of 100 web sites they have
visited and their experiences with those sites. In this case, we might select 10 of the sites at random out of
100, and present those selected sites to a respondent in random order. Designed correctly, random selection gives
all items equal
exposure to the respondents over the course of the research.
In
the days of paper-based scripts, rotations were done manually by
telephone interviewers, who would read a question’s choices in order
each time but start on a different choice for each respondent. Later,
computer-aided telephone interviewing (CATI) systems mimicked the manual
rotation process. Today, CustomerSat’s Web survey services handle true randomizations, as
well as rotations, with ease. For
more information, contact your CustomerSat representative or send
email to expert@CustomerSat.com.
CustomerSat.com
Announces Launch of CRMConnectTM Web Survey Solution and Contest
Enter to win a cool flat panel display,
compliments of CustomerSat.com! As a part of a recent launch of the
CRMConnectTM
Web survey solution, a drawing will take place among all registered
entrants. The CRMConnect
solution enables contact centers and help desks using any CRM system
to automate “connecting with” (surveying) customers as each case or
ticket is closed. The contest is simple—learn more about it and Ariba’s
use of CRMConnect
by visiting www.CustomerSat.com!
CustomerSat.com
500 Ellis Street., Mt. View, CA 94043
Mail
to: expert@CustomerSat.com
http://www.CustomerSat.com
(c)
2008 CustomerSat Inc
|
