Recently I have seen a number of discussions and posts discussing ways to follow-up on NPS surveys. Strangely, the writers seem to focus on transforming NPS, which is a quantitative survey with rigorous interpretation methodology into a qualitative interview, where insights are gained from reading customers’ comments or follow-up interviews to ‘drill down’ for the reasons behind the ratings. This accentuates the challenge with NPS results being non-actionable, and poses several additional problems we must think about and offer other methods to supplement the ratings.
First, let’s discuss the reasons for not using comments and interviews for extra insights from surveys:
- Sample size is always a concern for surveys. Comments are not always filled by customers, therefore the sample size is further reduced
- The manual nature of interviews will inevitably make scalability and cost a concern, further reducing sample size
- With global operations, language and time zones may eliminate a portion of the customers due to your inability to conduct interviews or correctly interpret comments
- Confirmation bias, where interviewers and comment readers only account for responses that confirm existing concepts, may pose a significant threat to the success of the survey program
- Discrepancies between comments and actual drivers of dissatisfaction are well documented. Relying on comments only will prevent you from confirming the comments via metrics
Now, should you read survey comments, or interview customers who rate you poorly? Absolutely, but do not confuse that for your main insights. Comments and interviews provide illustration to the broader conclusion you derive from researching the details of the survey and correlating them with your operational and demographic information.
So, how should you go about analyzing the responses from your NPS survey in greater detail?
First, by all means, call your customers back to follow up on survey results. Call those who rate you poorly, as well as those who rate you well. But, also correlate the results with demographic and operational data. For example, how does score vary across regions or industries? Do customers using a certain feature or function rate better or worse than others? Does your score vary based on support case count or their duration? How do events over time impact customers’ score? Last, and equally important, remember that non-responsive customers do that for a reason as well. Can you identify different factors that drive customers’ response rate? Does any of that indicate their propensity to renew, or churn?
In conclusion, go ahead and experiment with your customer survey results and the drivers behind them. Do not assume that what works for others will necessary work for you. If you have access to a person with statistics knowledge seek their help in building a regression model that identifies the impact of each factor on customer satisfaction. If you don’t, there’s much you can do on your own to analyze the results and understand your company’s specific environment and reach conclusions on what to improve next.