Derived Importance vs. Stated Importance

Over the years, the market research community has developed two primary ways to determine what’s most important to your customers:

• Stated Importance, as the name implies, asks them to state how important each attribute is.
• Derived Importance is determined by calculating the relationship between various attributes (independent variables) and an overall satisfaction or loyalty question (dependent variable).

Stated Importance

Stated importance can be the simple, obvious choice. After all, if you want to know something, why not just ask? Several different question formats may be employed:

• Rating: Give customers the opportunity to rate the importance of each attribute on a scale (e.g., 1–10) that’s used consistently throughout the questionnaire, so they are already comfortable with it. The assumption is that the customer will differentiate between the attributes, ascribing varying degrees of importance to each. An additional advantage is the ability to calculate a gap between stated importance and satisfaction.
• Ranking: Ask the respondent to rank the attributes or choices, with the most important first. This can be difficult if there are many choices. A variation on this approach is to ask the respondent to select the top three in importance. This makes sense when choosing from a list of factors that lead to a decision, such as selecting a vendor, visiting a web site, or attending a conference. A frequency distribution, expressed in absolute numbers and percent of the whole, quantifies the number of mentions across all the respondents.
• Constant-Sum Allocation: Ask each respondent to apportion a total of 100 points, for example, across the range of attributes. The idea here is to force a relative ranking—it’s a classic tradeoff exercise. This is effective only if the list of choices is relatively short. Also, our experience shows that it only works if the data collection method is visual, e.g., web-based, vs. oral, e.g., telephone.
• Open-Ended Questions: Pose questions that require more than a yes/no answer. This is the most challenging approach in terms of quantifying the relative importance of each, yet has the potential to uncover a new attribute.

Certain problems frequently arise when using stated importance to assess the relative importance of attributes:

1. Customers often find it difficult to differentiate, so, for example, they give high scores to every attribute. They may actually be subtly encouraged to give high scores, for fear that the company will fail to deliver on attributes that accrue lower scores. And, unless there are significant differences among attributes, the results are less useful. Indeed, there is general consensus in the market research industry that stated importance ratings offer a lower degree of credibility.
2. Because well-designed surveys should include all attributes that represent the customer’s experience with a company, factors will no doubt be included that are “givens” within a specific vertical, e.g., safety in the airline industry. It’s not surprising that safety would rate at the top of the scale, but it would not be very instructive for a vendor to base a customer satisfaction action plan around that attribute. Clearly, every airline is required to provide this; the key is to understand the less-obvious factors most important to the customer and differentiate based on providing the highest level of service in those areas.
3. In certain kinds of surveys (more typically related to consumer products), customers’ answers may be influenced by social norms or political correctness. For example, they might cite “quality” as the most important factor when selecting a product supplier, when in reality cost or convenience might be more important to them.

There’s another important factor affecting stated importance. The length of a questionnaire approximately doubles when you ask the importance of each attribute. Survey "real estate" is precious, and there is a constant trade-off between asking enough questions to yield valuable feedback and ensuring that the participant doesn’t drop out due to "respondent fatigue."

Derived Importance

The derived importance approach, by contrast, uses a less direct way of uncovering the factors that are most important to the survey respondent.

In some cases these factors are different from the ones the customer would directly cite as most important. Yet when acted upon, these factors are most likely to result in higher satisfaction.

This approach works by applying statistical models to the satisfaction/performance scores provided by the respondent. Statistical outputs, e.g., importance coefficients, indicate the level of importance. This conveys that the attribute shares variance with a dependent variable.

If, for example, the respondent scores a performance attribute an 8, and also scores the overall satisfaction question an 8, then there is a perfect correlation. If, conversely, the attribute received a 9 but the dependent variable received a 4, then there is a low correlation.

Pearson’s Correlation

Several different statistical approaches can be used to quantify the relationship of the independent variables and the dependent variable. We recommend Pearson’s Correlation, a straightforward approach in which the degree of correlation between each attribute and the dependent variable is conducted independently. Importance coefficients are generated that do not take into account multicollinearity (i.e., the degree that one attribute correlates with another attribute).

Although there may be some degree of correlation between attributes, the process of improving scores for each of the attributes may be totally different. This is why it is favorable to look at each of them separately.

For example, Time to Reach a Qualified Support Rep and Timely Problem Resolution will probably have some degree of collinearity. However, the two issues would require different processes in order to improve. For the former, an organization might consider improving its ACD phone system, increasing the number support reps, or improving the training of support reps. For the latter problem, remedies might include creating a knowledge base, installing an escalation process, increasing the number of agents, providing additional training, or re-allocating agents to handle peak times.

Kano Analysis

When implementing a short questionnaire, one approach—Kano analysis—is to compare an attribute’s relative (derived) importance to its stated (rated) importance. Derived importance is computed via regression weights, obtained when regression is done on each attribute against overall satisfaction. Stated importance is determined from the questions in the survey, by asking the importance of each attribute.

When asking for stated importance and calculating derived importance, the results can be plotted on a quadrant chart, as in the example:

• Top Right Quadrant: These attributes have both a high stated importance and high derived importance. They have the highest importance.
• Top Left Quadrant: With a low stated importance and high derived importance, these are unspoken motivators. They don’t stand out as particularly important in the customer’s mind, but effort applied here will have a great impact on overall satisfaction.
• Bottom Left Quadrant: These attributes have low stated importance and low derived importance. They are not good indicators of satisfaction.
• Bottom Right Quadrant: With a high stated importance and low derived importance, these are core items, the bare minimums expected by customers.

Summary

When determining what actions to take, the key is to understand the intersection of important attributes and performance shortfalls. Because an organization typically has limited resources to effect change, it is critical to correctly assess the former. Identifying strategic advantages (high importance, high satisfaction), which should be maintained, and key vulnerabilities (high importance, low satisfaction), which are prime opportunity areas, allows the organization to implement appropriate actions that will yield the biggest bang for the buck.