One of our early clients was a Fortune 100 company that made capital equipment.  They had been routinely conducting a telephone based customer satisfaction survey for years and were looking for a new vendor when we happened to run across them.   

The company had historically conducted a survey every six months, the results of which were the primary contributor to the calculation of executive management bonuses.  The principle reason they were shopping for a new vendor was a case of what I’ll call “deliverables fatigue”.  Twice a year they were being delivered a three-volume set of reports, with each volume roughly the size of a Manhattan telephone directory.  These reports were so overloaded with techno-babble filler that no one in the company had time to even try to read them, let alone glean an understanding of what they said.

Naturally, our first meeting was devoted to introducing the company to our methodology and its advantages, as well as the attendant deliverables.   Compared to what they had been receiving for years, the deliverables – built on simplicity and clarity - were an instant hit.   However, there was one point of concern.

Telephone based surveys are notorious for producing inflated results.  The Cassandra Phenomenon is a major contributor there, as is the fact that phone surveys, which are highly intrusive on busy schedules and almost universally disliked, produce rushed answers as respondents try to get it over with as soon as possible.  Because our approach mitigates both problems, the result is responses that are not inflated by biases and outside influences.     

So the issue was that the company’s bonus system had been built around artificially increased scores, which meant that a change in methodology would upset the entire bonus apple cart (their words).  Better yet, the people charged with internal administration of the survey were part of the bonus pool.  It’s safe to say that euphemisms like “inmates guarding the asylum”, “fox guarding the henhouse” and “tail wagging the dog” were all brought to mind. 

A compromise was eventually worked out wherein we agreed to continue survey activities via telephone while some sort of normalization formula was sought and implemented.  The initial thought was to continue for a single six-month survey cycle while the troops were prepared for the change.  To make a long story short, four six-month cycles came and went.  Management procrastinated for two full years, at which time we finally transferred the contract to a sister company and walked away.  There are a host of research firms out there who conduct phone surveys.  We did not want to be one of them, even if it meant disassociating with a Fortune 100 company.  There are also a host of reasons why we do not do business with some companies, but to the best of my recollection, this was the one and only case where producing more accurate results actually stood as an impediment.   

This all started in 1996.  To this day, the company is still conducting phone surveys.  I have no idea how many millions in bonuses have been paid out in the intervening years. 

Sadly, these are not the only executives I’ve run into out there for whom customer satisfaction scores are a game.  While most of our clients view customer satisfaction as a business imperative, others are what I call score mongers.  If they don’t like the top line scores, they’ll add in lower tier scores and create a composite to make things look better.  If they don’t like the scores for a particular performance category one year, they’ll just drop the questions next year and throw in some fluff questions instead.  For them, the name of the game is to generate whatever will look the best – or in at least one case, whatever will line their pockets the best – instead of devoting the time and energy required to produce meaningful change.  Lots of smoke, lots of mirrors, very little substance.   

Sounds an awful lot like the mentality that’s so pervasive in Washington, DC, doesn’t it?                            

A major software company with a customer list that reads like the Fortune 500 had been conducting a “home-grown” web-based customer satisfaction survey for years.  Though they didn’t get much of a response rate, they were nonetheless pleased with results that showed that 94% of the respondents were “satisfied” with the company’s performance. 

There was just one problem.  Notwithstanding the lofty scores, one of the two major revenue streams of the company was declining. 

In late 2005, fearful that the results they had been relying on were not accurate, the company commissioned a multi-national customer satisfaction survey employing the InfoQuest Business Process Review.  When the results of that survey came back, it showed that only 14% of their customers indicated they were “Totally Satisfied”.  It instantly became clear that they had been living with a sense of false security produced by data that was inaccurate, less than candid and imprecisely defined.   

What changed?  First, the InfoQuest response rate was over 12 times higher.  That not only brought key decision makers into the process, but it removed the unavoidable influence on accuracy that low response rates commonly produce.  In other words, the silent majority was finally heard, and their views were vastly different from those of the previously vocal minority. 

Second, when customers think, even suspect, that their identity is or may be known (and access codes to web surveys raise such questions), nearly 70% of the population will not voice a strongly negative opinion out of concern that having their name attached to such opinions may produce unwanted ramifications (the Cassandra Phenomenon).  For that reason, customers typically feel more comfortable expressing candid opinions to a third party than they do directly to a business partner.   

Third, the InfoQuest survey focused on the most critical measure of customer satisfaction – the percentage of totally satisfied customers.  There is no ambiguity there—those customers are the only ones not susceptible to competitive approach.

There are two discussion points that arise here. 

First, not all surveys are created equal.  Badly worded questions, methodologies that produce biases, and poor response rates all contribute to survey results that may bear no resemblance to reality.  Having “numbers” is of no value if they aren’t accurate.      

Second, even when they are accurate, numbers can be manipulated.  For marketing and public relations purposes, it’s not at all uncommon for companies to group together the scores of anyone who is not dissatisfied, and then proudly proclaim to the world that the rest – often an impressive number – are “Satisfied”.  While for purely external presentations, such proclamations make for good press (even if you’re deceiving the public, it’s a manipulation that‘s generally harmless), the monumental mistake that far too many companies make is to believe their own press. 

If you look at our May 5 discussion on the relationship between customer satisfaction and revenue, it’s been proven that Totally Satisfied customers contribute, on average, over two and half times the revenue of a Somewhat Satisfied customer. 

Let me say it again!  OVER TWO AND HALF TIMES THE REVENUE. 

So when you take those two groups and add them together to create a composite “satisfied” score, sure, the result looks better to the media and the stockholders, but in reality, what you are doing is utterly delusional.  Take our software company as a prime example.  While they spent years patting themselves on the back for a 94% “satisfaction” score, their sales were declining.  

As a footnote, the company was sold off and eventually dismantled less than a year later.  By the time they figured out they had it all wrong, it was too late.