Customer Satisfaction Measurement - The Leadership Factor

Significance testing is soooo 20th century

I'm just in the middle of an excellent book called Beyond Significance Testing. It's one of an increasing number of works to point out the flaws in both the fundamental theory and conventional usage of the "classic" significance test.

Let's have a quick look at how significance testing works. Imagine we want to see if there is a significant difference between the Satisfaction Index for men and women in the situation below:

How do we know if it's significant? The first thing is to work out what question we're asking, which is "does this difference exist in the population as well as in the sample?".  Then we create a null hypothesis, which is that there is no difference between men and women. Finally we conduct a hypothesis test, a t-test in this case, which tells us the chances of finding a difference as big as this in the sample assuming the null hypothesis (that there's no difference in the population). This value, the p. value, conventionally needs to be less than 5% for us to reject the null hypothesis...telling us that there is very unlikely to be no difference between men and women in the population (note that it doesn't tell us how big the difference is!). Clear as mud? I thought so.

So if this approach has flaws, which it does, what should we do instead? The alternative is to look at effect size and margin of error. In other words, instead of asking our stats package for an obscure p. value, let's ask it how big the difference is likely to be in the population...which is what we're really interested in.

One quick way to do this is to plot the margin of error, or confidence interval, of the Satisfaction Indexes straight onto the chart. This has a number of advantages: it is much easier to explain and understand, it illustrates the uncertainty of the measurement, and it captures the size of the difference rather than a simple "significant" or "not significant".

A better test of a specific question like this is to work out the margin of error of the difference, which in this case is 3.1 ±1.8. In other words we can conclude with 95% confidence that the difference between men and women in the population is between 1.3 and 4.9.

Much more powerful, and much easier to understand, than null hypothesis significance testing isn't it?

Note from a satisfied defector

Mobile phone customers in the UK are a famously fickle bunch and not to let the side down, I've just changed my mobile phone supplier.

In the context of Stephen's comments below, I'm very much a satisfied defector. I've been with the Orange network for over 5 years and I've been generally happy with their service. So what's prompted the change, have they committed some massive blunder on my account and I've walked away because of that? The answer is no, quite simply I wanted a particular phone that isn't available with Orange so for that reason - I've moved to another operator.

As Stephen suggests, like many defecting customers, I may well move back to Orange if it suits me at some future point. I'd still recommend them, but they aren't right for me at the moment. Perhaps if they'd tried a little harder to hold onto me I'd have stuck with them, or accepted some alternative offer, but they didn't.

What are you doing in your organisation to hold onto your satisfied customers?

If satisfied customers defect....can defecting customers be satisfied?

Let's come back to those satisfied customers who defect. We tend to assume, because of the strong link between satisfaction and loyalty, that defecting customers are dissatisfied with us. If we accept the idea that satisfied customers defect, then some defecting customers must be satisfied. That must be worth thinking about.

What are the implications? Firstly, and most importantly, a satisfied defector is a customer who is not lost forever...yet. Handling the defection process well is the last chance we have to influence how a customer feels about us. Nothing we do should turn that customer into a dissatisfied defector.

Recently I signed up for a free trial of one of the leading online DVD rental services. The service was okay, I had one or two problems with faulty discs, but overall I was quite satisfied. When the trial came to an end I decided that it wasn't, on balance, worth continuing at the normal rate. So from their point of view I had signed up in response to an introductory offer and was now defecting, costing them money. In monetary terms I was a terrible customer but, importantly, I was reasonably satisfied with them. I would have considered signing up for the service at another time, and I would have said quite positive things about their service to other people.

That all changed when I went through the process of cancelling my subscription. There was no option to cancel online (by contrast to the super-easy online sign-up process!), you had to phone up. That's an irritating policy, and the reasons are transparent, so I was a little bit riled when I did call them. After twenty minutes of the most frustrating telephone conversation ever, including obnoxiously worded threats about returning DVDs on time or being charged, I was seething. But I did finally manage to cancel my subscription.

This company has recognised defections as a problem, probably because of its strategy of accepting a high cost of acquisition in the free trial, but it deals with the problem in precisely the wrong way. Instead of trying to manage my attitude about the company they tried to directly impede my behaviour by making it difficult to defect. In doing so they turned me from a satisfied defector, a potential source of profit, to a virulent antagonist.

Never try to keep your customers hostage. If they want to leave, make the process as pleasant and easy as possible. You will make priceless attitudinal gains, and that defecting customer is far more likely to use you in the future and say good things about you.

Why do satisfied customers defect?

I was interviewed recently by a journalist who asked me what I thought about the classic "flaw" in customer satisfaction measurement—satisfied customers sometimes defect. I find it bizarre that anyone thinks this is strange.

Humans are very unpredictable. They also, in most markets, have a range of good options to choose from. To cap it off, many suppliers are so focused on acquisition that they offer special rates or other incentives for new customers. No wonder customers defect, even when they're fairly happy.

More interesting is the fact that customers can, with effort, be made loyal. Anyone unsure of the benefits of customer satisfaction could spend a valuable 5 minutes pondering how that might happen! But that's a post for another day.

What I want to talk about today is the distinction between attitudes and behaviours. Hidden behind the buzzword of the day (commitment, engagement etc.) is the basic idea that customers have feelings about you, feelings which may be positive or negative, strong or weak. The reason we care is that these feelings have a strong influence over their behaviour, and some behaviours (retention, recommending, buying other products) are good for business.

The misconception that I started this post with is based on expecting these attitudes and behaviours to tally perfectly—they don't, and we shouldn't expect them to. Nonetheless, having a customer base which has positive attitudes about you will lead to greater sales across the whole customer base, even if the odd customer acts unexpectedly.

So, why do satisfied customers defect? Not because satisfaction is a poor predictor of loyalty, but because:

• People are innately unpredictable AT THE INDIVIDUAL LEVEL. The average behaviour of groups IS predictable.
• Passive satisfaction is not enough to guarantee loyalty—there are a lot of good options in the market.
• Companies often spend more effort on acquisition than retention, encouraging disloyalty.

 

Mystery shopping best practice – facts not judgements

In the European Union there are restrictions on the use of mystery shopping that prevent it being used for disciplinary purposes against individual employees. It is increasingly recognised by good employers that mystery shopping is best used for factual rather than judgemental aspects of service and to provide positive feedback and recognition to employees. Good companies also understand that it provides operational information rather than a reliable measure of how satisfied or dissatisfied customers feel.

Treating suppliers fairly

Wally Ollins writes more compellingly about brand than anyone else I know. One of the cornerstones of his view of the world is that a successful brand is one that feels the same wherever you touch it.

His point is that the way a company behaves, it's style if you like, should flow from a consistent and powerful set of values that permeate the business. The same basic values should govern the way the business relates to everyone—employees, customers and even those on the other side of the relationship, suppliers and partners.

It would be a terrible mistake to think that you can pretend to be a caring company when dealing with customers while treating your employees badly, though some organisations still seem to believe this is possible. I'll let Branson have the defining word here:

Customers don’t come first. Employees come first. If employees are treated right then service will follow.
Richard Branson

Likewise, it is inconsistent to say you want to build long-term relationships with customers while screwing your suppliers down to a price that puts them out of business. Consistency extends both ways in your relationships, and suppliers are stakeholders just as much as customers and employees. This is fundamental to having a business that is really about creating more value for everyone. It's also the only way to preserve your all-important reputation.

One of the best examples of this in practice is Innocent drinks, who impressed me deeply when I interviewed them for this Stakeholder Satisfaction case study[PDF]