And speaking of where’s the beef -

A couple years ago, portions of the business world jumped through hoops and danced with glee after the release of a book called The Ultimate Question. The basic premise of the book was that in order for any business to grow and prosper, all you need to know is your customer’s responses to one question - How likely are you to recommend us to a friend or colleague? After subjecting responses to that question to a manufactured scoring mechanism of unknown origin, the result was given the name Net Promoter Score, or NPS for short.

Fred Reichheld, who seems to have an umbilical association with an outfit called Bain Consulting, wrote the book. Bain, as anyone who has been around a while knows, is one of those upper crust, one might say prestigious consulting firms; the sort that boasts a blue-chip client list and is rumored to have a fee structure that would rival the gross domestic product of most of the third world and parts of the second. Maybe throw in Canada for good measure. And Detroit.

So NPS was born, packaged into a book, and presented to the multitudes as the second coming of customer satisfaction. From what I can tell, the initial reception was somewhere between tepid and lukewarm. But then something big happened and the tide suddenly turned. And turned in a big way.

Jeff Immelt, CEO of General Electric, reportedly stood up one day and said something along the lines of, “Hey! This is good stuff, this NPS. I like it. In fact, I like it so much that from this day forward, I’m going to use it in my company.” Or words to that effect.

Well, old Fred, (and presumably the folks at Bain, at least one publishing house, a few bookselling chains, and probably some paper and ink companies), must have damn near had organisms when those words were spoken. After all, an endorsement from a guy like Immelt, head of one of the most successful companies in the history of the Milky Way and maybe a few other galaxies, is to book sales what hitting 5,000 homes runs is to a baseball player at contract time. Just like that, an obscure little theory shook of the dust and jumped into the spotlight. Like lemmings to the sea, people started jumping on the NPS bandwagon.

Now, we do a lot of surveys, and we talk to and work with a lot of companies. The NPS theory had been floating around out there for a while, and though it had been typically dispatched with a snide chuckle or two, back in ’06 and early ’07, interest in it took on a life of it’s own. I finally reached the point of having little choice but to look into it in some detail. I never did find the beef, but I sure did find a couple of other interesting things. Those findings led to an article entitled Net Promoter Score - Search for the Magic Pill, which I invite you to pay a visit to. While that visit will give you the full laundry list of things about NPS that made my jaw drop, I’ll devote this space only to the more, um……attention-getting…..aspects of Mr. Reichheld’s work. Specifically,

1) By his own admission, in terms of being able to predict customer behavior, using responses to a single question is not as accurate as using responses to dozens of questions, like everyone else does. That revelation, which is about as surprising as the notion that the sun rises inthe east, leads to a rather obvious question……..why do it?

Ready for this?

2) The apparent motivation in not asking a full array of questions is that “most customers in this busy world won’t give you that much time – witness typical survey response rates from 2% to 20%” – and

3) ……”you couldn’t afford the surveying and data processing expense if they did“.

Well, imagine my surprise. I was shocked, shocked I tell you, to learn that after conducting surveys for nearly 15 years, typically with anywhere from three to five dozen questions, and while generating an average response rate of over 70%, (oh, and for a price that does not come close to the GDP of anyplace on the planet), that what we’ve been doing can’t be done.

But wait. It can be done. It has been done. It’s still being done.

Was it possible, I wondered, that a certain degree of legerdemain might be at work here? Maybe. I know for certain that if I ever produced a 2% response rate, or even a 20% response rate for that matter, I’d be wearing a disguise, traveling under an assumed name, and operating out of a remote location for fear that if I showed myself, angry villagers carrying pitchforks and torches would find my office and burn it to the ground. And me with it.

Or is something else at work here? I pondered that one long and hard, and after six, maybe seven seconds of deliberation, it finally hit me.

Let’s say for the sake of argument that you decide to go out and conduct a NPS survey. You’ll pay someone some decent bucks to collect the data, tally it up, and present you the results. What if, to your dismay, the score falls far short of where you want it to be, or where you though it would be? Now what do you do? What actions do you take to raise the score, to better your company, to make your customers more loyal?

Ah, paradox. Irony. Conundrum.

You didn’t bother to ask what your customers think of your service, or product quality, or technical support, or billing, delivery, phone systems, communication efforts or any of the dozens of other things that comprise and define a customer relationship. So, the Ultimate Question, the one thing you need to know? Aside from the fact that its own creator has cautioned you that it’s not all that accurate, it’s also about as far as you can get from being complete. It’s not even close to being the one thing you need to know. How could it be?

So, where’s the beef?

One, I bet most of the people on the NPS bandwagon bought a book. Ka-ching!

Two, I’ll bet nearly all of them bought more survey work. Probably a lot more. Ka-ching, ka-ching.

And three, just a guess here on my part, but when that time came, I’d be willing to bet there was a prestigious consulting firm lurking in the wings, ready and eager to provide that little service. Ka-ching, ka-ching, ka-ching, ka-ching……..well, you get the idea.

Bottom line. I see NPS as a brilliant piece of marketing. Diabolical in its simplicity, perverse in its appeal, and with more up sell potential than even Pentagon contractors enjoy, from a marketing standpoint, it’s a work of art. But at the end of the day, despite all the sizzle, I’m not seeing much of a steak.

What do you folks think? Anyone out there eager to bless it as the Ultimate Question? Anyone been taken to the cleaners by any prestigious consulting firms?

Let me know what you think.

One of our early clients was a Fortune 100 company that made capital equipment.  They had been routinely conducting a telephone based customer satisfaction survey for years and were looking for a new vendor when we happened to run across them.   

The company had historically conducted a survey every six months, the results of which were the primary contributor to the calculation of executive management bonuses.  The principle reason they were shopping for a new vendor was a case of what I’ll call “deliverables fatigue”.  Twice a year they were being delivered a three-volume set of reports, with each volume roughly the size of a Manhattan telephone directory.  These reports were so overloaded with techno-babble filler that no one in the company had time to even try to read them, let alone glean an understanding of what they said.

Naturally, our first meeting was devoted to introducing the company to our methodology and its advantages, as well as the attendant deliverables.   Compared to what they had been receiving for years, the deliverables – built on simplicity and clarity - were an instant hit.   However, there was one point of concern.

Telephone based surveys are notorious for producing inflated results.  The Cassandra Phenomenon is a major contributor there, as is the fact that phone surveys, which are highly intrusive on busy schedules and almost universally disliked, produce rushed answers as respondents try to get it over with as soon as possible.  Because our approach mitigates both problems, the result is responses that are not inflated by biases and outside influences.     

So the issue was that the company’s bonus system had been built around artificially increased scores, which meant that a change in methodology would upset the entire bonus apple cart (their words).  Better yet, the people charged with internal administration of the survey were part of the bonus pool.  It’s safe to say that euphemisms like “inmates guarding the asylum”, “fox guarding the henhouse” and “tail wagging the dog” were all brought to mind. 

A compromise was eventually worked out wherein we agreed to continue survey activities via telephone while some sort of normalization formula was sought and implemented.  The initial thought was to continue for a single six-month survey cycle while the troops were prepared for the change.  To make a long story short, four six-month cycles came and went.  Management procrastinated for two full years, at which time we finally transferred the contract to a sister company and walked away.  There are a host of research firms out there who conduct phone surveys.  We did not want to be one of them, even if it meant disassociating with a Fortune 100 company.  There are also a host of reasons why we do not do business with some companies, but to the best of my recollection, this was the one and only case where producing more accurate results actually stood as an impediment.   

This all started in 1996.  To this day, the company is still conducting phone surveys.  I have no idea how many millions in bonuses have been paid out in the intervening years. 

Sadly, these are not the only executives I’ve run into out there for whom customer satisfaction scores are a game.  While most of our clients view customer satisfaction as a business imperative, others are what I call score mongers.  If they don’t like the top line scores, they’ll add in lower tier scores and create a composite to make things look better.  If they don’t like the scores for a particular performance category one year, they’ll just drop the questions next year and throw in some fluff questions instead.  For them, the name of the game is to generate whatever will look the best – or in at least one case, whatever will line their pockets the best – instead of devoting the time and energy required to produce meaningful change.  Lots of smoke, lots of mirrors, very little substance.   

Sounds an awful lot like the mentality that’s so pervasive in Washington, DC, doesn’t it?                            

A major software company with a customer list that reads like the Fortune 500 had been conducting a “home-grown” web-based customer satisfaction survey for years.  Though they didn’t get much of a response rate, they were nonetheless pleased with results that showed that 94% of the respondents were “satisfied” with the company’s performance. 

There was just one problem.  Notwithstanding the lofty scores, one of the two major revenue streams of the company was declining. 

In late 2005, fearful that the results they had been relying on were not accurate, the company commissioned a multi-national customer satisfaction survey employing the InfoQuest Business Process Review.  When the results of that survey came back, it showed that only 14% of their customers indicated they were “Totally Satisfied”.  It instantly became clear that they had been living with a sense of false security produced by data that was inaccurate, less than candid and imprecisely defined.   

What changed?  First, the InfoQuest response rate was over 12 times higher.  That not only brought key decision makers into the process, but it removed the unavoidable influence on accuracy that low response rates commonly produce.  In other words, the silent majority was finally heard, and their views were vastly different from those of the previously vocal minority. 

Second, when customers think, even suspect, that their identity is or may be known (and access codes to web surveys raise such questions), nearly 70% of the population will not voice a strongly negative opinion out of concern that having their name attached to such opinions may produce unwanted ramifications (the Cassandra Phenomenon).  For that reason, customers typically feel more comfortable expressing candid opinions to a third party than they do directly to a business partner.   

Third, the InfoQuest survey focused on the most critical measure of customer satisfaction – the percentage of totally satisfied customers.  There is no ambiguity there—those customers are the only ones not susceptible to competitive approach.

There are two discussion points that arise here. 

First, not all surveys are created equal.  Badly worded questions, methodologies that produce biases, and poor response rates all contribute to survey results that may bear no resemblance to reality.  Having “numbers” is of no value if they aren’t accurate.      

Second, even when they are accurate, numbers can be manipulated.  For marketing and public relations purposes, it’s not at all uncommon for companies to group together the scores of anyone who is not dissatisfied, and then proudly proclaim to the world that the rest – often an impressive number – are “Satisfied”.  While for purely external presentations, such proclamations make for good press (even if you’re deceiving the public, it’s a manipulation that‘s generally harmless), the monumental mistake that far too many companies make is to believe their own press. 

If you look at our May 5 discussion on the relationship between customer satisfaction and revenue, it’s been proven that Totally Satisfied customers contribute, on average, over two and half times the revenue of a Somewhat Satisfied customer. 

Let me say it again!  OVER TWO AND HALF TIMES THE REVENUE. 

So when you take those two groups and add them together to create a composite “satisfied” score, sure, the result looks better to the media and the stockholders, but in reality, what you are doing is utterly delusional.  Take our software company as a prime example.  While they spent years patting themselves on the back for a 94% “satisfaction” score, their sales were declining.  

As a footnote, the company was sold off and eventually dismantled less than a year later.  By the time they figured out they had it all wrong, it was too late.