A major software company with a customer list that reads like the Fortune 500 had been conducting a “home-grown” web-based customer satisfaction survey for years.  Though they didn’t get much of a response rate, they were nonetheless pleased with results that showed that 94% of the respondents were “satisfied” with the company’s performance. 

There was just one problem.  Notwithstanding the lofty scores, one of the two major revenue streams of the company was declining. 

In late 2005, fearful that the results they had been relying on were not accurate, the company commissioned a multi-national customer satisfaction survey employing the InfoQuest Business Process Review.  When the results of that survey came back, it showed that only 14% of their customers indicated they were “Totally Satisfied”.  It instantly became clear that they had been living with a sense of false security produced by data that was inaccurate, less than candid and imprecisely defined.   

What changed?  First, the InfoQuest response rate was over 12 times higher.  That not only brought key decision makers into the process, but it removed the unavoidable influence on accuracy that low response rates commonly produce.  In other words, the silent majority was finally heard, and their views were vastly different from those of the previously vocal minority. 

Second, when customers think, even suspect, that their identity is or may be known (and access codes to web surveys raise such questions), nearly 70% of the population will not voice a strongly negative opinion out of concern that having their name attached to such opinions may produce unwanted ramifications (the Cassandra Phenomenon).  For that reason, customers typically feel more comfortable expressing candid opinions to a third party than they do directly to a business partner.   

Third, the InfoQuest survey focused on the most critical measure of customer satisfaction – the percentage of totally satisfied customers.  There is no ambiguity there—those customers are the only ones not susceptible to competitive approach.

There are two discussion points that arise here. 

First, not all surveys are created equal.  Badly worded questions, methodologies that produce biases, and poor response rates all contribute to survey results that may bear no resemblance to reality.  Having “numbers” is of no value if they aren’t accurate.      

Second, even when they are accurate, numbers can be manipulated.  For marketing and public relations purposes, it’s not at all uncommon for companies to group together the scores of anyone who is not dissatisfied, and then proudly proclaim to the world that the rest – often an impressive number – are “Satisfied”.  While for purely external presentations, such proclamations make for good press (even if you’re deceiving the public, it’s a manipulation that‘s generally harmless), the monumental mistake that far too many companies make is to believe their own press. 

If you look at our May 5 discussion on the relationship between customer satisfaction and revenue, it’s been proven that Totally Satisfied customers contribute, on average, over two and half times the revenue of a Somewhat Satisfied customer. 

Let me say it again!  OVER TWO AND HALF TIMES THE REVENUE. 

So when you take those two groups and add them together to create a composite “satisfied” score, sure, the result looks better to the media and the stockholders, but in reality, what you are doing is utterly delusional.  Take our software company as a prime example.  While they spent years patting themselves on the back for a 94% “satisfaction” score, their sales were declining.  

As a footnote, the company was sold off and eventually dismantled less than a year later.  By the time they figured out they had it all wrong, it was too late.

This entry was posted on Friday, May 15th, 2009 at 11:18 am and is filed under 3. Playing With Numbers. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.