Customer Satisfaction Measurement - The Leadership Factor

The Holy Grail

At our conference this year, I talked about the Holy Grail of customer attitude measurement—proving the benefit by showing the financial returns from high levels of satisfaction, loyalty, etc.

This is important because, if we want the customer experience to be taken seriously at board level, we need to join the grown-up conversation. We need to talk pound signs (or dollar signs, or Euros).

My main message was that it's dangerous to see this goal as a Holy Grail. Mythologising the quest for proven links to financial outcomes makes it seem impossible, attainable only by the chosen few—the Galahads of customer service and business intelligence.

My view is that Percival is a much better Grail knight for us to model our quest on. He only had to ask the right question when he had the chance.

So what is the right question? Ultimately what we want to know is “if I invest £x in improving satisfaction, what will the payoff be in improved profit?”. But couched in those terms, it's an impossible question for most businesses to answer. Does that mean we should give up? No.

I think we can make big strides towards answering the question simply by looking at the implicit causal links embedded within it, and breaking the question apart to make them explicit. Ulimately we are looking at a three step causal chain:

1. If I invest £X* , how does that change our performance?
2. How does our performance affect customer perceptions?
3. How do customer perceptions affect their behaviour (in ways that improve profit through increased sales or reduced cost)?

I'll go into more detail in a follow-up post, but in a nutshell I think we can answer those by:

1. A combination of guesswork, experience, and experiment.
2. Well targeted event-driven surveys linking satisfaction to specific, tangible, experiences and behaviours.
3. An aggregate based on all the bits and pieces of the jigsaw we can lay our hands on. (e.g. How much do complaints cost to handle x how many more complaints we get from dissatisfied customers, what does one inbound call cost x number of extra calls for dissatisfied customers, acquistion cost x difference in actual retention rate)

So I think that most organisations could get their hands on the Holy Grail, or at least a convincing estimate of it, if they approach the quest in the right way. But do we have any evidence that the journey is worth it?

Results from January's UKCSI, which we conduct on behalf of the Institute of Customer Service, show that there is such evidence. The full analysis is available on the ICS website (read the executive summary as a good starting point), but one of the key charts is this one. It shows the strong link between customer satisfaction in the Retail sector and sales growth. Amazon, John Lewis, and other retailers generating great customer experiences, are growing faster than their less successful competitors.

Perhaps it's time you started finding your own path to the Holy Grail?

 

*********

* Or make changes which don't require investment!

Truth is another story

This post in a nutshell: stories are powerful, but dangerous.

Powerful, because they are the best way to persuade people to do what we want them to do (journalists, lawyers and salespeople know this well). Dangerous, because we also use them to sell easy explanations to ourselves.

 

Fooled by ourselves

In The Black Swan, Nassim Nicholas Taleb talks about the "Narrative Fallacy"—our tendency to believe stories we've made up to explain facts. Pretty much every newspaper story about scientific research is guilty of this (see for example this post by Kaiser Fung).

The problem is that we tend to be happy with an explanation which is plausible, as long as it is consistent with the evidence (particularly if it fits with our preconceptions), whilst forgetting that there are lots of other potential explanations. By jumping to conclusions we automatically cut off any further investigation of the real explanation, so this process tends to be self-fulfilling.

A key part of the scientific method is designed to tackle exactly this. Proper testing is all about thinking up ways our story could be shown to be wrong, not piling up more evidence that it is right. Sadly the testing part of the process rarely transfers into business.

Science: Facts->Hypothesis->Testing

   vs.

Business strategy: Facts->Hypothesis....... 

So what happened to fact-based management? It has been a victim of three fatal weaknesses:
1) Proper scientific testing of causality is difficult, sometimes impossible.

2) Science is a bit nerdy, and the people who run businesses were more often the captain of the school rugby team than they were the chess club.

3) Business likes to perpetuate a myth of itself as slightly macho, seat of the pants, instinctive and passionate. This is all very well if you want an organisation staffed entirely with Apprentice contestants (shudder), but doesn't help if you want to make good decisions.

Proper science is the only way to find out what actually works. As I've said before on this blog, why not put some of your assumptions to the test?

 

The responsibilities of the story-teller

If we have a duty to ourselves, and to our businesses, not to be fooled by our own glib explanations and preconceptions, we should also do our best to make sure that we don't lead others down the garden path.

It may not always feel like it, but our stories can be just as compelling for other people as they are for us. We need to be very confident what we're saying is true. The cynic in me thinks that many people would be paralysed with terror if they thought there was the slightest chance that what they were saying might actually change the way anything was done...but shouldn't we at least act as if it might?

Stories are, without doubt, the best way to make people take notice of what you're saying. But before you rush off to create your next story, make sure it's grounded in evidence. You don't necessarily need to show your working, but you do need to have done it. Great writers spend most of their time researching and rewriting—shouldn't you do the same?

10 second survey: what one thing would you change?

Quick survey using Google forms. Please take part, and pass it on! I'll report any interesting findings here.

Improving customer satisfaction, part 1

I'm going to be doing a series of posts about improving customer satisfaction, based on lessons we've learnt with our clients. A gentle start today, with three maxims to guide our thoughts.

Image source: http://www.flickr.com/photos/gettysgirl/3400383586

Finding out what customers want is sometimes less than half the battle. Not only do you have to get from understanding to implementation, you often need to deal with the fact that gearing your product or service to an average customer can mask a range of underlying requirements. A standard approach risks dissatisfying almost everyone.

There is no average customer.

We can address this problem with good staff or through tailoring, but it's also well worth finding out the impact of over or under delivery. Often the impact is not symmetrical. We asked one client's customers whether they were contacted “too much”, “about right”, or “not enough”. Opinion was divided—does that make it an impossible situation? Not when you know that customers who were contacted too much were as satisfied as those who thought it was about right. Only customers who weren't contacted enough were unhappy.

Goldilocks notwithstanding, “too cold” and “too hot” are not equally bad.

The lesson we learned with that client is applicable to most organisations. Active* contact with customers is almost always a good thing, improving customer satisfaction and often reducing the number of inbound calls. Using SMS and email is, increasingly, an attractive way to keep customers informed (guess how many people have that as one of their priorities for improvement!) while keeping cost under control. 

You can't talk to customers too much...(unless you're selling)

All of which could be distilled to "find out what customers want and give it to them".

* I refuse to say "proactive"!

Satisfaction measurement - should you go deep or broad?

At the WCCF conference this week there was quite a lot of talk about the advantages and disadvantages of various ways of measuring the customer experience, which I'll probably post more about next week.

One of the things that struck me about what some organisations are doing, particularly with NPS, is that the focus is incredibly strong on getting very frequent data at the lowest possible level (even down to individual call centre agents).

This kind of granularity can be a very powerful tool for change, as it gives people a sense of ownership and accountability for the score, and of course could be used with any metric—there's nothing special about NPS in that regard.

The flipside is that NPS, or to be fair any survey that you can realistically run at these frequencies and volumes, tends to give you very shallow data. You get a score, and perhaps a verbatim comment, but no quantitative insight or specific qualitative information into the reasons behind it.

 

The choice here, it seems to me, is whether you should go for deep or broad understanding of the customer experience. For many organisations the ideal answer will be a combination of both, with some sort of automated (web, email or IVR) option to take the “pulse” of the organisation at a granular level combined with a detailed telephone interview survey backed by thorough analysis and insight.

A ten step guide to getting high satisfaction scores

People often act as if the aim of a customer satisfaction survey is to get a high score, or hit a target, rather than to improve how customers feel about the organisation. Here is our 10 step guide to faking a high satisfaction score without the hard work of satisfying customers.

1. Ask customers about things you know you do well
In order to measure satisfaction properly you have to use the “lens of the customer” rather than the “lens of the organisation”. This partly means language, partly means customers not caring about your internal hierarchies and processes, but most importantly means which aspects of the relationship you measure.

If you come up with your own questionnaire, without involving customers, then by definition you will be asking customers about the areas you focus on. These are likely to be things you are good at, but they may not be the most important things to customers.

The [Building Society] has sent her a customer satisfaction questionnaire. Does she like the decor of her local branch? How long does she have to wait in line? Do staff smile? She has responded positively to each question. There is no opportunity to do anything else.

Suddenly she writes at the bottom, “But I hate you.”
Geoff Ryman – 253, a novel for the Internet

2. Use a positively weighted scale
After all, we know that customers are mostly satisfied, right? So we must need more positive response options. Let's go for the old “Excellent, Good, Fair, Poor” and count everyone who doesn't say “poor” as satisfied. Organisations focusing on their customers rather than the score might make the mistake of opting for a long enough balanced scale to pick apart the scores of satisfied customers, but that's dangerous. What if they all score us 6 out of 10?

3. Force customers into a score you can misinterpret
Well apparently the board don't like that scale. Someone seems to have read a book that said it was “biased”. Oh well, at least if we go for a short enough verbal scale we know the vast majority will give us a “courtesy 4”. Never mind whether that would be a 6 or 9 on a ten point scale, we can still count them as satisfied, and that's what makes the press release.

4. Pretend that “satisfied” is a good thing
“Satisfied” is such a useful word. On the one hand it sounds pleasingly positive—“90% of customers are satisfied”, excellent. And yet at the level of the individual customer, it's so easy to make people satisfied—all you have to do is meet their basic needs and expectations. After all 6 out of 10 is satisfied.

5. Only talk to happy customers
It's much more fun to talk to the customers who like us, why don't we ask them to do the survey for us? They know us much better, after all. Anyway, you shouldn't really worry too much about what Bill says—he's always complaining. Can we delete that score? Excellent.

I think the actions of almost all marketers say, “we’d rather you were happy, but if you can’t be happy, please go away.”
Seth Godin

6. Find an easy benchmark to compare yourself against
This one is 100% bullet proof. No one in your organisation will ever question that your market is completely different from every other market (and yes, you do have the most demanding customers). That means that no benchmark from outside your market is fair, particularly if you don't emerge well from the comparison. The really clever bit is that you can define your market exactly how you want to—genius!

7. Use mystery shopping or internal KPIs as measures of satisfaction
This is like an extreme form of technique number 1. By measuring yourself not only on the things you think are important, but using your own criteria, you can almost guarantee a good result. Companies that were not just focused on getting a high score might use these tools in addition to a measure of customer attitudes, rather than in place of it.

8. Pick your timing
Just made a big change? Don't survey.
Something went wrong? Don't survey.
Rainy day? Don't survey.
England just won the World Cup? Fill your boots.

9. Make sure they know you're watching
Offer to help customers with anything they don't understand on the questionnaire. Stand over their shoulders as they fill it in with the pen you've lent them. Why not offer to fill it in on their behalf, after all it is a bit of a hassle isn't it? Make sure you understand any reasons for dissatisfaction by challenging any low scores, preferably in a slightly aggressive way.

10. Bribery
Offer customers an incentive for filling out the survey—if it's good enough the warm glow should carry over to the scores they give. It's a shame you can't make the incentive conditional on good scores, really (though see point 9).

All of these things are much easier than the prospect of, gasp, actually improving the products or services we offer customers. You may feel that I've ladled the sarcasm on rather thickly here, but you come across these mistakes on a regular basis in current satisfaction measurement. Many organisations take the path of least resistance and adopt the comforting appearance of a high score they don't deserve.

Companies that do this are living in a fool's paradise, and they'll be rudely shaken out of it as initiatives like the UKCSI increasingly provide a published, objective, and stringent measure of customer satisfaction. If you're serious about measuring and improving customer satisfaction you need a tough measure. Find out more in our Improving Customer Satisfaction course.

10 tips for presenting data that will drive action

I take both a personal and a professional interest in presentation. Getting a presentation right means your audience will leave with the right message, understand what the data is telling them, understand what they should be doing, and be inspired to take action.

Presentation is key, and the ultimate test of success or failure is not a ripple of polite applause and a kind word from the Chief Exec, but whether your business changes as a result.

I believe that the main factor which determines the success of a presentation is the clarity of the message, both on individual slides and in the presentation as a whole. This is doubly true of numerical data, such as customer survey results, where it is easy to get bogged down in slide after slide of analysis.

When it comes to presenting numerical data, here are 10 key principles that I have stolen or evolved over the years. These form the basis for most of the recommendations I make on my course.

1. Show me the numbers
Simplicity comes from clarity of message, not dumbing down. If you hide the numbers then I have to trust you, and frankly I don't. By all means use a R2G colour scheme to make the message clearer, but numbers are not innately scary and it's important that they are there for those that want them.

2. Order by something
There is almost always an order more interesting than simply listing items alphabetically or in the way they were presented on the questionnaire. Think about the story the data is trying to tell and then sort it in a way that supports that (e.g. largest values first, largest changes first, largest differences from average first etc).

3. Let the data breathe
Get hold of the Visual Display of Quantitative Information, or something similar, and have your eyes opened to the amount of “chartjunk” and wasted ink in the average table or chart. Fade chart mechanics such as grid lines and axes into the background by changing them from black to grey. You will soon find that you're obsessing about allowing the data room to stand on its own, rather than decorating it. “If the numbers are boring then you've got the wrong numbers.”

4. Tell the truth
This might seem obvious, but it is easy to fool yourself as well as your audience, particularly when evidence can be made to conform to your preconceptions. Make sure you are asking yourself tough questions, and that you can back up your assertions with evidence. I recommend the Halo Effect as an effective antidote.

5. If it's boring, ditch it
When we do a piece of analysis, or ask a question in a survey, it is often because we're wondering if something interesting will emerge. Often it doesn't. If so, I recommend you leave that question or analysis on the cutting-room floor—padding out your presentation with analysis that has no implications for the future dilutes the message and makes you look dull. Have the courage to make decisions about which analysis is meaningful.

6. You are responsible for the actions that are taken
This is a scary one. Using the wrong piece of analysis, or using it in an inappropriate way, can lead to bad decisions. Quadrant charts are notorious in this regard. This can lead to real, damaging, effects in the way your organisation performs. Be careful, and make sure you are confident that you have got everything right. If you want people to take your analysis seriously, you have to take it seriously too.

7. Don't create work for your audience
There is no need to dumb down for your audience, but there is also no reason to be obstructive. Avoid abbreviations, jargon and other barriers. Subtler sins include using poor typography, such as words rotated by 90° in order to fit. This can almost always be avoided, and doing so is a sign of respect for your audience.

8. Analysis and reporting are separate functions
This links in with point 5. Not everything that happens at the analysis stage should end up in the reporting/presenting phase. This is upsetting, since you may want to demonstrate how much hard work you did, but crucial if you want to leave a clear message.

You may also choose to present messages in a different way, or using different charts, than the tools you used to discover them in the first place. A factor score coefficient matrix is unlikely to mean much to the board, but the message about the existence of common factors underlying the items on the questionnaire is perfectly comprehensible.

9. Delete first, then add
Perhaps the easiest way to achieve number 3 is to remove the unnecessary ink that Excel and other programmes tend to add to graphs. Clearing away the clutter, then fading the chart mechanics into the background, will free up lots of empty space. This is a good thing in itself, but it also allows you the room to add elements which help the message stand out. Don't be afraid to annotate directly onto your charts where it helps focus attention on the message you're trying to tell.

10. Find a message, then sell it
The single most important thing is that you know what the analysis is trying to say. A good discipline when preparing presentations is to write “headline style” titles. By using strong declarative statements (“Satisfaction is up in the North West”) rather than category headings (“Regional satisfaction vs. 2008”) you are forced to think about what the message is, as well as making a much more engaging presentation. If you can't find a headline, then see point 5.

Looking across these headlines should allow you to see a building story that will leave the audience in no doubt about what your main message(s) are. If not you need to decide what they are and restructure the presentation to deliver them clearly.

How scales work

Bear with me, this will get relevant!

The world of climbing has, somewhat oddly, been riven with controversy over scales. One of the UK's top climbers, James Pearson, made a first ascent of a very hard and dangerous new route which he named “Walk of Life”. So far, so good.

He decided to give the route a grade of E12 which, without getting into the technicalities of the British trad grading system, would make it the hardest climb in the world. Many armchair critics were sceptical about the grade, and the situation got even more heated when Dave MacLeod repeated the route and gave it a grade of E9 (still very hard and dangerous, mind you).

 

[Image source: http://davemacleod.blogspot.com/]

The poorly-informed and ill-tempered debate that followed in the climbing media (which both Dave and James rose above) focused on whether the grading scale is “broken”, or whether climbers give grades in order to attract publicity/make themselves look hard.

I think this is missing the point completely. The real lesson is that it is impossible for humans to give a completely objective measurement of anything. This is similar to the objection that “one person's 8 is another's 9” or, for that matter, “one person's satisfied is another's very satisfied”.

It's never a good idea to rely too much on a single score given by a single person, but the law of large numbers means that, over a sufficient number of people, average scores will stabilise and be reliable. This is the magic of quantitative surveying—averaging lots of unreliable scores gives us a reliable average.

Interestingly, the few sensible comments about the Wall of Life grading debate draw attention to the fact that climbing grades are always established by consensus. This is simply an informal version of the same phenomenon.

Analysing qualitative data

Delegates on my training course often ask me about analysing qualitative data, usually in the form of verbatim comments from customers. Compared to quantitative data, qualitative data often seem bulky and time-consuming—surely there must be an easier way?

Sadly there isn't, really. There is no software with a magic “code all this data” button, and even if there were it would be a drastically bad idea to use it. This is an area in which a human brain needs to be employed for any real insights to emerge. Mind you, so is quantitative analysis...but that's a post for another day!

There is software designed to make your life easier when it comes to analysing qualitative data. Collectively these programmes are known by the stunningly unwieldy acronym CAQDAS*. If you want a full list you can find one at the ASC website (caution—ugliest website ever). Prominent names include NVivo, Qualrus and The Ethnograph.

While they can't do the analysis for you, computer programmes do give you tools that make your life easier. You can get your computer to produce a word frequency count and concordance (key words in context), as well as to search for key phrases. But this kind of automated work is rarely very interesting.

CAQDAS software really comes into its own in easing the process of building up a coding frame and then using that to generate theory. Many CAQDAS programmes are built around an academic approach known as Grounded Theory, which is great for inductive reasoning in your PhD thesis but, frankly, probably overkill for your customer satisfaction survey.

Whatever software tools you use, effective analysis of qualitative data will always take time and care. Coding is a back-and-forth process that takes frustratingly longer than you expect. Above all you need to remember that coding can take you away from the depth of the underlying data—the codes are there to help you think, not to convert your data into arbitrary quantifiable categories!

* Computer Assisted Qualitative Data Analysis Software or sometimes QDAS or QDA for short!

Zen and the art of customer satisfaction measurement

It's tempting to over complicate the things that are important to you. This means that when we sit down with clients to design a satisfaction survey (important to us) to measure the feelings of their customers (important to them) there's often more pressure to add  than to subtract.

This comes from a laudable desire to understand customers in great detail, but risks alienating customers with the very tool that is supposed to be getting to the bottom of their feelings.

That's why we go on and on about the importance of exploratory research. A customer survey must be based on questions which are relevant to customers, and reflect the things which they're thinking about when they make their judgements about you. Frequently these things are painted with a broad brush, but that's how customer attitudes work. No matter how much we would like them to be precise and actionable, customer attitudes will always be impressionist rather than photorealistic.

The zen art of customer surveys is to accept this occasional imprecision as the price for an accurate measure, and to find ways round it from the point of view of actionability. Our main weapon here is collecting verbatim comments to illustrate the individual stories behind each low score. This balances the broad-brush measurement, that reflects the way customers see the world, with the precision and actionability an organisation needs to improve. Over-complicating the list of requirements to be scored only frustrates the customer and spreads a single impression across five apparently different areas.