One of our early clients was a Fortune 100 company that made capital equipment.  They had been routinely conducting a telephone based customer satisfaction survey for years and were looking for a new vendor when we happened to run across them.   

The company had historically conducted a survey every six months, the results of which were the primary contributor to the calculation of executive management bonuses.  The principle reason they were shopping for a new vendor was a case of what I’ll call “deliverables fatigue”.  Twice a year they were being delivered a three-volume set of reports, with each volume roughly the size of a Manhattan telephone directory.  These reports were so overloaded with techno-babble filler that no one in the company had time to even try to read them, let alone glean an understanding of what they said.

Naturally, our first meeting was devoted to introducing the company to our methodology and its advantages, as well as the attendant deliverables.   Compared to what they had been receiving for years, the deliverables – built on simplicity and clarity - were an instant hit.   However, there was one point of concern.

Telephone based surveys are notorious for producing inflated results.  The Cassandra Phenomenon is a major contributor there, as is the fact that phone surveys, which are highly intrusive on busy schedules and almost universally disliked, produce rushed answers as respondents try to get it over with as soon as possible.  Because our approach mitigates both problems, the result is responses that are not inflated by biases and outside influences.     

So the issue was that the company’s bonus system had been built around artificially increased scores, which meant that a change in methodology would upset the entire bonus apple cart (their words).  Better yet, the people charged with internal administration of the survey were part of the bonus pool.  It’s safe to say that euphemisms like “inmates guarding the asylum”, “fox guarding the henhouse” and “tail wagging the dog” were all brought to mind. 

A compromise was eventually worked out wherein we agreed to continue survey activities via telephone while some sort of normalization formula was sought and implemented.  The initial thought was to continue for a single six-month survey cycle while the troops were prepared for the change.  To make a long story short, four six-month cycles came and went.  Management procrastinated for two full years, at which time we finally transferred the contract to a sister company and walked away.  There are a host of research firms out there who conduct phone surveys.  We did not want to be one of them, even if it meant disassociating with a Fortune 100 company.  There are also a host of reasons why we do not do business with some companies, but to the best of my recollection, this was the one and only case where producing more accurate results actually stood as an impediment.   

This all started in 1996.  To this day, the company is still conducting phone surveys.  I have no idea how many millions in bonuses have been paid out in the intervening years. 

Sadly, these are not the only executives I’ve run into out there for whom customer satisfaction scores are a game.  While most of our clients view customer satisfaction as a business imperative, others are what I call score mongers.  If they don’t like the top line scores, they’ll add in lower tier scores and create a composite to make things look better.  If they don’t like the scores for a particular performance category one year, they’ll just drop the questions next year and throw in some fluff questions instead.  For them, the name of the game is to generate whatever will look the best – or in at least one case, whatever will line their pockets the best – instead of devoting the time and energy required to produce meaningful change.  Lots of smoke, lots of mirrors, very little substance.   

Sounds an awful lot like the mentality that’s so pervasive in Washington, DC, doesn’t it?                            

This entry was posted on Tuesday, May 26th, 2009 at 11:36 am and is filed under 3. Playing With Numbers. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.