Category Archives: Surveys

NPS Again, This Time With Feeling?

Recently I came across two interesting posts discussing NPS®. Each of them seems to miss one, or more, important points about creating a sustainable customer survey program that actually produces tangible results

First, and very interestingly, is Fred Reichheld, creator of NPS, on linkedin telling us to Stop Thinking Like a CEO (and Think Like a Customer Instead). If we think about the title for a minute we’ll realize that unlike the customer, a CEO has responsibility for identifying weaknesses in the customer experience and taking action to fix them. In his post, Mr. Reichheld tells the stories of two CEOs who transformed their companies’ customer experiences. But, did those CEOs do what Mr. Reichheld is asking us to do? No, they did not. They never stopped thinking like a CEO, but they did bring the customer perspective into the organization, and, most importantly, acted on it to deliver a better customer experience

In his post Mr. Reichheld claims that:

“The best approach is to ask customers on a scale of zero to 10 how likely they would be to recommend your products or services. This is what […] companies do every day to be loyalty leaders. By closing the loop with their customers and taking action on the reasons why customers love doing business with them or not […]”

But while the scale is detailed, the need to “take action” remains nebulous – how do we know what actions to take, and how do we verify their impact?

Another post that caught my eye last week was on the Bluenose blog, titled Driving Net Promoter Adoption: 3 Must Do’s. In his post Don MacLennan, Bluenose CEO, states that leadership commitment, organizational alignment and customer follow through are essential to increasing a company’s NPS results, and he recommends engaging with customers as a survey follow-up in order to establish the cause of their dissatisfaction and eliminate it

Some readers may ask whether there’s anything wrong about engaging with customers to find out what’s causing dissatisfaction. The answer is that there’s nothing wrong about that, but when we think about a process driven business we can’t rely on anecdotal evidence as the sole input to our improvement efforts, and that for several reasons:

  1. The plural of anecdote is not data – so no matter how many customers the company interviews they will be only a small portion of the customer base and their opinions remain anecdotal rather than a complete picture
  2. Discrepancy between stated and actual behavior drivers is a well known phenomena and is documented in numerous academic papers
  3. The challenge in converting anecdotal evidence into a sustained organizational effort to improve products and services

With that said, what would be the best way to follow up on NPS, or any other, survey and generate sustained improvement

The answer should be no surprise to the blog’s followers. Survey responses, in correlation with your operational data, provide the organization with very clear insights on actual (as opposed to stated) customer behaviors and their drivers. With these insights it is possible for the organization to develop sustained improvement efforts based on the most critical elements of the customer experience, and gauge their impact on customer satisfaction, as well as customer behavior. It’s doable, and usually easier than we tend to think, but does require commitment and a change in the way we think

Net Promoter, NPS, and the NPS-related emoticons are registered service marks, and Net Promoter Score and Net Promoter System are service marks, of Bain & Company, Inc., Satmetrix Systems, Inc. and Fred Reichheld

How to not ask for feedback

Earlier this week I had the chance to participate in a seminar. It was an interesting day in a beautiful location, and lunch was provided. Part of lunch a bag of chips which contained this text:

IMG_1231

It’s possible the manufacturer had every intention to solicit all feedback from customers. But, my first reaction was “what number should I call if I don’t love your chips?”

I am sure many of us have similar stories about such interactions, from the car dealership asking us to “fill the survey only if we can give them 9 or 10, otherwise call the manager” to this bag of chips. While these are amusing stories they have a valuable lesson – you get what you ask for, and if all you want is positive feedback that’s what you are most likely to get.

Having said that, let’s remind ourselves of the value in customer feedback. First, and usually the case of these entertaining messages, is reconfirmation – we’d like the customers to confirm for us that we are doing a good job. Second, and most importantly, is that systematic customer feedback helps us understand where our operation is failing to deliver the expected product or service. If we don’t solicit that feedback we’ll never get it and eventually lose to those who do and continue to improve their operation based on that.

Minimalistic Surveys?

Green hexadecimal computer code fading to the right

Recently I had to contact Coursera support to ask a question. A day later I received an email message asking for my impressions:

screenshot_172

Clicking on the link in the message takes the user to a text box for additional comments.

Given my interest in surveys, and using them for insight and improvement, I found this message thought provoking. There are several benefits to using such a short survey:

  • Reduce survey fatigue – fewer questions will drive higher response rate
  • Binary choice – customers’ opinion is very clear

However, there are a few downsides to this type of survey:

  • Lacking nuance – gaining improvement insights from a single binary choice requires extensive additional processing and analysis to correlate results with operational metrics. Lack of rigorous analysis will drive over reliance on text comments
  • Interpretation bias – reading text comments in an attempt to gain systemic insight presents a danger of several biases. Eloquent comments will naturally be given more weight, and comments in foreign languages depend on the availability of someone who can read and translate them

With these points considered, can enterprise technology companies enjoy the benefits of shorter surveys without sacrificing the quality of insight produced? To answer that, we need to look at two distinct points:

  • What’s the shortest survey that will provide the information required to gain insights into the most beneficial improvements?
  • Can the company analyze text comments and maximize information gained out of those?

Finally, how can companies make shorter surveys work?

I believe that in order for these surveys to work well and provide meaningful insights several conditions must be fulfilled:

  • Sufficient volume of cases and responses
  • A wealth of operational data to correlate to survey responses
  • Rigorous analytic process, including text analysis (remember your non English speaking customers), and no, reading the comments for broader insights is not rigorous