Monthly Archives: May 2015

Minimalistic Surveys?

Green hexadecimal computer code fading to the right

Recently I had to contact Coursera support to ask a question. A day later I received an email message asking for my impressions:

screenshot_172

Clicking on the link in the message takes the user to a text box for additional comments.

Given my interest in surveys, and using them for insight and improvement, I found this message thought provoking. There are several benefits to using such a short survey:

  • Reduce survey fatigue – fewer questions will drive higher response rate
  • Binary choice – customers’ opinion is very clear

However, there are a few downsides to this type of survey:

  • Lacking nuance – gaining improvement insights from a single binary choice requires extensive additional processing and analysis to correlate results with operational metrics. Lack of rigorous analysis will drive over reliance on text comments
  • Interpretation bias – reading text comments in an attempt to gain systemic insight presents a danger of several biases. Eloquent comments will naturally be given more weight, and comments in foreign languages depend on the availability of someone who can read and translate them

With these points considered, can enterprise technology companies enjoy the benefits of shorter surveys without sacrificing the quality of insight produced? To answer that, we need to look at two distinct points:

  • What’s the shortest survey that will provide the information required to gain insights into the most beneficial improvements?
  • Can the company analyze text comments and maximize information gained out of those?

Finally, how can companies make shorter surveys work?

I believe that in order for these surveys to work well and provide meaningful insights several conditions must be fulfilled:

  • Sufficient volume of cases and responses
  • A wealth of operational data to correlate to survey responses
  • Rigorous analytic process, including text analysis (remember your non English speaking customers), and no, reading the comments for broader insights is not rigorous