Are transactional NPS surveys dead?

Perhaps a better question than the one in my headline would be “Should transactional NPS surveys be killed off?” Let me discuss this in some detail, then suggest action. Spoiler alert: I believe using NPS for transactional surveys is usually a bad idea, at least the way most companies currently do so.

What am I talking about?

Transactional surveys are those that take place immediately after some sort of interaction with a customer. The one that jumps to most people's attention when thinking about this is a support survey, meaning a survey that you receive just after you have resolved a problem with someone you have spoken to on the phone. You are probably used to getting them after you have your car serviced, for example. Airlines often send them after a flight. These are all examples of transactional surveys and are quite different from a product survey or an overall relationship survey. Double-blind competitive loyalty surveys are something else again. I think you get the picture.

What's the biggest issue?

Allow me to state what I believe is the biggest issue: Transactional NPS does not measure company NPS. NPS is quite easy to understand as a number and I believe the majority of business managers can explain how it is calculated. What it represents is a different story. We now see many companies that report NPS in their annual reports, and also in quarterly investor calls. The issue is that they never seem to say where the number has come from.

Having investigated some of these, I think I can say with confidence that the majority of NPS scores we see reported are for call centers, and not the overall company. Let's face it, for most companies, the vast majority of customers have no contact with a call center in any given year. This means that the NPS number being reported for the company is meaningless. A positive call center NPS trend does not correlate with future corporate growth. Again, transactional NPS simply does not measure company NPS. Only overall company trends do so. Unfortunately, this is not subject to financial reporting regulation, so I cannot say this is corporate fraud, even if I think it is.

There is an even bigger issue than this, but I will save it for the end of this article. Feel free to guess what it is in the meantime.

Any other problems?

Well, there is the general issue that most companies use transactional NPS results simply to populate scorecards, rather than to take action. Regular readers may remember that I did two exercises wherein I responded to 100 surveys and documented what the various companies told me they had learned and what they were going to do. The first time, only British Airways wrote to me. The second time, nobody did, even though I deliberately sent zero scores with strong complaint wording to about 20% of them.

Companies also send you surveys when you don't know whether you are getting what you need. For example, you may have had something happen and have called your insurance company to initiate the process of getting someone to assess damage and to pay for repairs. But you may get a survey after the initial phone call and you have no idea how the rest of the process is going to work.

And of course, don't get me started on the surveys car sales people send you after you interact with them. David Mingle, Executive Director, North American Customer Experience for GM explained what is going on at a public event a few years back. Sales people typically need to average over 95% satisfaction in the GM dealer surveys to be able to participate in their bonus scheme. One low score can require ten perfect scores to even it out. This is why they pester you to give you a 10, and even ask you not to take the survey unless you can give them a 10. My more general point here is that transactional surveys scores are quite easy to rig.

Transactional NPS can be perfectly OK

There is a specific type of situation in which transactional NPS surveys are perfectly OK, and there is a general situation where transactional surveys work well. Note that I left NPS out of the second category.

Transactional NPS works well and predicts future purchases and recommendations well in situations where the transaction represents most of the experience you have with the company. eCommerce companies like Zappos are a great example. Most of your experience is in finding and ordering the clothing or shoes you want. Once you confirm your order, you receive an MüS survey request. That's fine as the discovery, selection, and ordering process represents about 80% of your Zappos experience. Delivery, unpacking and a possible returns process represent the other 20%.

Transactional surveys can work well

The generic situation in which transactional surveys work well is where you can act on them. Nevertheless, in almost all cases, I believe you should drop NPS and use Customer Effort Score, as making things easy for customers is the objective of most transactions. A reasonable alternative is a customer satisfaction score for that transaction. You should also use Natural Language Processing to analyze the answers to open questions, rather than relying solely on unreliable humans. So yes, I believe transactional NPS should be dead, in general.

And finally, here is ‘the sting in the tale’

There is a gigantic issue with transactional surveys of all kinds. It's quite simply to understand. If you get a 5% response rate, you are learning nothing at all that is specific to 95% of your customers. I think you should be able to understand that this is really bad. And please remember that the 5% are the just ones who feel strongly enough to take your survey. Others may simply and silently have given up on you.

I strongly believe you should replace transactional surveys by a combination of what I call Customer AI possibly together with NLP. Regular readers know that continuous predictive customer analytics mean you use the thousands of operational data points you have for each and every customer to predict NPS and future purchase and renewal probabilities. If you also have records of email and chat interactions, transcripts of call center interactions, indeed, just about anything involving text, you will find researching causes behind both desirable and unwanted data trends to be considerably easier. NLP will also give you the customer quotes and stories you need to accelerate approval of improvement projects.

Conclusion, and a question

In conclusion:

1. Transactional NPS should be killed off.
2. CES and CSAT are usually more meaningful metrics for transactional surveys and using them avoids confusion with overall product and brand-level NPS numbers.
3. Technology has changed rapidly over the last few years and you are now far better off using an AI solution that covers all customers as your main form of customer analytics. Occasional brand-level surveys can nonetheless help identify brand image and other emotional items that can be important in consumer businesses.

It should be reasonably clear that I now believe that there are few situations where surveys are the best possible solution. But what do you think? Do some of my arguments above make no sense at all? What do you recommend? Should we kill off transactional NPS surveys?

Notes

OCX Cognition predicts customer futures. Our breakthrough SaaS solution, Spectrum AI, lets enterprises transform what’s possible in customer experience. Reduce your customer risk, break down silos, and drive speedy action – when you can see what’s coming, you can change the outcome. Building on more that 15 years of CX-focused expertise, we’ve harnessed today’s advances in AI, elastic computing, and data science to deliver on the promise of customer-driven financial results. Learn more at www.ocxcognition.com.

Maurice FitzGerald is a retired VP of Customer Experience for HP's $4 billion software business and was previously VP of Strategy and Customer Experience as well as Chief of Staff for HP in EMEA. He and his brother Peter, an Oxford D.Phil in Cognitive Psychology, have written three books on customer experience strategy and NPS, and a fourth book that focuses on Peter's cartoon illustrations for the first three. All are available from Amazon.