Customer Satisfaction Measurement - The Leadership Factor

Scatterplots—they're really not that hard!

The intermittently good blog Junk Charts has another post highlighting the danger of thinking that you can analyse the relationship between two variables simply by plotting them next to each other on a line chart.

The problem is that line charts really don't give you the ability to tell if there is a relationship or not, but your brain may well con you into believing that they do because of our tendency to see patterns (even if they're not there). Take this chart:

Is there a relationship between the two lines? When you've made up your mind click here to see a scatter plot of the same data. It's a lot easier to tell isn't it?

Why are people so afraid of scatterplots? They're one of the most useful basic tools in any analyst's toolkit, and they're really a very simple idea—essentially a scatterplot is just a map. If you don't feel comfortable with them perhaps it's time you learnt to be!

Show me the data

Nicholas Bissantz has another great post over at his blog. He returns to a favourite theme of his (and mine)—simplifying data for greater clarity is all very well, but sometimes it's best to show the actual numbers.

Averages can hide a wide range of actual performance. With large data sets, like surveys, we address that with statistical tools such as variance, standard deviation and confidence intervals. But what do you do when dealing with data that isn't statistical?

Bissantz suggest graphing the data points as well as the average, a technique that I've found very useful with small samples:

His example makes the point nicely—three very different real-world situations that would produce the same average, and therefore the same conclusion for an unwary analyst.

Designing Questionnaires - Art or Science?

I've just received yet another badly thought out questionnaire today and it makes me wonder why. Why are so many questionnaires poorly constructed with bad layouts, poor choice of scaling and my main irritation - badly worded questions?

Perhaps part of the problem is the ease with which you can construct online questionnaires now, with a profusion of services that make it easy (too easy?) to ask the wrong kind of question. It's like having a powerful car but no sense of direction or map - you aren't going to get where you want to be! Of course, it isn't just online questionnaires that are poorly designed, there are lots of print examples out there too. I suspect that many of the people that issue printed questionnaires have never tried to complete their own survey. It isn't just the design elements that I've mentioned above, they'd realise that there isn't actually enough space to write a response in or that printing of that type of paper makes the inks smear and so on.

So, given that a well designed questionnaire can boost response rates and provide a whole level of more detailed insight, what things should you consider when designing one: Start with:

• Sampling - who are you sending it to, how many responses do you need?
• Appearance - instructions and question design
• Questions - open, closed, scale etc.
• Layout - what route will they follow through the questionnaire etc.

If you're looking for more ideas and advice, you might want to look at this course or visit the website and pose us a questionnaire question!

Classic charts...and beer

Interesting article in the Economist looking at three of "history's best" charts.

Ever since Tufte published the Visual Display of Quantitative Information*, Charles Minard's graphic of Napoleon's march on Moscow has been the standard example of a thoughtful and compelling "infographic". I even use it in my course...despite its somewhat tangential relationship to Customer Satisfaction Measurement!

Playfair is a fair choice too. He invented, for better or worse, many of the graphic forms that we are familiar with today, including the bar chart, line chart and even the dreaded pie chart. The Economist shows an early attempt to make political mileage out of charts.

The third person featured is Florence Nightingale, who is sometimes thought to have invented the pie chart, but didn't. She did come up with the "Nightingale Rose", or polar area chart, which is the one covered by the Economist. Frankly this is an odd choice, as it is not one of history's best charts by any means. Nonetheless the outcome of the analysis was of great importance, forcing a review of the sanitary conditions of army barracks and hospitals at a time when disease killed far more soldiers than enemy action.

For similar reasons, my preferred third choice would have been John Snow's map of cholera deaths in Soho.

A map that forced the closure of a lethally infected water pump (the Broad Street pump) and finally began to convince people that cholera was water-borne and not spread by smell.

Next time you're in Soho or Carnaby Street, find your way to Broadwick Street, as Broad Street is now known, and have a pint in the John Snow pub. It seems an ideal way to commemorate the closing of an era in which drinking beer was safer than drinking water.

* You can read our review of VDQI here[PDF]

A puzzle

Imagine you are a manufacturer of consumer electronics, specialising in MP3 players. You measure customer satisfaction once a year, and you split the results to show the scores for each of your three MP3 players: a large one, a small one and a tiny one. Customer satisfaction is highest with the tiny player and lowest for the large player.

The good news is that, since last time, satisfaction has risen with all three players. Great...but hang on — the overall satisfaction index for the business has fallen!

What on earth is going on? This situation is one that can easily trip organisations up, particularly when they have significant customer segments, as in this case, or business units. Is this organisation getting better or worse? Your challenge for next time

• Can you explain what on earth is going on?
• What could this organisation have done to avoid this paradox?