Customer Satisfaction Measurement - The Leadership Factor

Brand tags

Every so often the blogosphere comes up with a good idea. Once in a blue moon it's a really good idea.

Brand image has always been a tricky beast to pin down, but most people would define it as something like the sum of all the attitudes and associations that consumers hold about a particular brand. Some different ways of putting this:

It has always seemed to me that your brand is formed primarily, not by what your company says about itself, but what the company does. Jeff Bezos, CEO, Amazon.com

Consumers build an image [of a brand] as birds build nests. From the scraps and straws they chance upon. Jeremy Bullmore

A brand is a living entity—and it is enriched or undermined cumulatively over time, the product of a thousand small gestures. Michael Eisner

We have the power to shape brands to be what we want… Wally Olins

A brand image is shaped by the products and services themselves, by the consumption environment, by marketing communications and, perhaps most importantly, by the behaviour of the organisation and its employees.

So how do you measure something so vast and intangible? One technique is to ask people what associations they have for a particular brand, something along the lines of "what 3 words come to mind", and over a sample of hundreds some interesting patterns will emerge.

Brand tags is a website that does exactly the same thing, across a whole host of brands, and represents the results in the form of a "tag cloud" similar to the one to the right of this blog. A fascinating concept, and there are some revealing results. You can take part in the tagging or simply browse through the  brands.

Here's a snippet of the cloud for a famous sports shoe brand:

Can you guess who it is? If not peek at the image title/alt tag, or listen to more hip hop!

Public sector customers?

I received an interesting email today from the Head of Customer Development at a large local authority. He asked whether the different relationship that public sector suppliers have with "customers" should dictate any differences in approach.

My answer was that, although the different relationship entails some surface changes to approach, the core remains very much the same. It means, for example, that there is little point in asking conventional loyalty questions like "will you still be a customer" or even the ubiquitous "would you recommend", but customer satisfaction is still very relevant.

The starting point, whatever your organisation, should be to think about what your desired outcomes are. In the private sector, the company may want increased customer retention, higher sales, larger market share, higher profits and so on.

What are the equivalent desired outcomes for the public sector? Once those have been pinned down it will be much easier to see how to approach both service delivery and customer research. For instance you might set goals based on reducing complaints, or increasing satisfaction as an end in itself, rather than as a step on the road to greater profits.

What do you think—are public sector "customers" fundamentally different from private sector customers? Should public sector organisations manage their customer relationships differently?

Designing Questionnaires - Art or Science?

I've just received yet another badly thought out questionnaire today and it makes me wonder why. Why are so many questionnaires poorly constructed with bad layouts, poor choice of scaling and my main irritation - badly worded questions?

Perhaps part of the problem is the ease with which you can construct online questionnaires now, with a profusion of services that make it easy (too easy?) to ask the wrong kind of question. It's like having a powerful car but no sense of direction or map - you aren't going to get where you want to be! Of course, it isn't just online questionnaires that are poorly designed, there are lots of print examples out there too. I suspect that many of the people that issue printed questionnaires have never tried to complete their own survey. It isn't just the design elements that I've mentioned above, they'd realise that there isn't actually enough space to write a response in or that printing of that type of paper makes the inks smear and so on.

So, given that a well designed questionnaire can boost response rates and provide a whole level of more detailed insight, what things should you consider when designing one: Start with:

• Sampling - who are you sending it to, how many responses do you need?
• Appearance - instructions and question design
• Questions - open, closed, scale etc.
• Layout - what route will they follow through the questionnaire etc.

If you're looking for more ideas and advice, you might want to look at this course or visit the website and pose us a questionnaire question!

Research...it's not brain science

The New York Times has received a bit of a roasting over this piece (published a couple of weeks ago) using brain imaging (fMRI) to draw conclusions such as:

When we showed subjects the words "Democrat," "Republican" and "independent," they exhibited high levels of activity in the part of the brain called the amygdala, indicating anxiety.

This is nonsense. More to the point, it's such obvious nonsense that the piece should never have made it to print, which would have saved the NYT the embarrassment of this scathing response and an enthusiastic pile-on from the blogosphere—Ben Goldacre's piece was where I picked up the story, and Language Log has a guest post from the astonishingly distinguished Martha Farah thoroughly eviscerating the original.

Neuroscience is fascinating, but it's also particularly prone to abuse and pseudoscience. Merely showing pictures of the brain to people has been shown to make them more prone to accept flawed explanations...in other words "brain scans indicate" is much more persuading than "researchers think". Even though that's basically the same thing. The research behind this is summarised in a Language Log post, or you can read the journal article[PDF] in press.

All of which should make us very sceptical whenever someone claims to have done anything useful with "neuromarketing". Interestingly, the best blog I know of that claims to deal with neuromarketing has precious little brain imaging, but rather a lot of well-designed traditional experiments. We're not going to be stopping people in the street and asking them to stick their heads in a fMRI scanner any time soon.

Which, on balance, is probably a good thing.

The elephant in the room

A mini rant this time, on a topic that's usually swept under the carpet.

Delegates on our training courses often get confused by the term "sample" because the same word is used for both the number of questionnaires you send and the number you get back. Why? Because in the examples given in most textbooks, the two numbers are the same.

If you turn to an introductory statistics or research text, the tacit assumption in almost every case is that the sample achieved is 100% of those who were invited. These books then skip merrily on to statistical estimation of confidence intervals, margins of error, significance tests and so on. Fun!

In practice much the same thing happens—analysts happily apply the formulas given in those textbooks to their data. Formulas that tell them about the accuracy of their data and their ability to measure things out there in the real world.

Great, except for one thing—response rates aren't really 100% are they? Which means you may have a problem with non-response bias. It also means that the standard statistical formulas cannot give a perfect measure of how precise your data are. In theory, if you don't get a near perfect response, you can't project the findings of your survey to the population you're interested in.

So what do we do? We get the response rate as high as we can and hope for the best. But getting a good response is really, really important. So much so that I would argue response rate is actually more important than sample size.

All of which boils down to a very simple suggestion—sometimes it would be worth sacrificing sample size for a better response rate. In other words, if you are suffering from a lack of response it would be better to use a more expensive, but more effective, technique like telephone or even face to face interviewing rather than self-completion methods.

Bizarrely this will give you less apparent reliability, since the formulas don't take response rate into account, but it will immensely improve the robustness of your survey. As a rule of thumb aim for at least a 40% response rate, preferably 50%, and don't pretend you can't see the elephant in the room.

76% of Surveys are 'Voodoo Polls' (or are they?)

Has the proliferation of of DIY survey tools improved the standard of surveys that we see in the media? Probably not seems to be the answer. Seth Godin's recent post on surveys raises some interesting points. He identifies four kinds of survey:

- Census Surveys
- Public Surveys
- Professional Surveys
- Census-based Analytics

I think the type we see most in the media is the public survey which typically quotes a "58% of listeners think..." type of statement. Certainly, surveys are more and more popular in the media and as the basis of press releases. In itself, that isn't a bad thing. What has to be avoided though is confusing these voodoo polls* with a carefully structured, scientifically designed survey sent to a representative sample of your audience.

It's worth considering that although the necessity of basing the results of a CSM survey on a representative sample of customers is widely acknowledged, the technical aspects of doing so are little understood and often neglected, making the survey unreliable and not much better than a 'voodoo poll'.

*Voodoo polls is the term commonly used to refer to the voluntary sort of surveys used on the TV and radio - 'text in and let us know what you think'. This type of survey notoriously suffers from unrepresentative samples and is generally unreliable.