in customer research

Storytelling requirements (and why most focus groups are a waste of time)

So, Malcolm Gladwell got me thinking about focus groups the other day. Actually, he got me thinking about the characteristics of groups and the way that people perform when in front of their peers, as well as perfect strangers.Malcolm was kind of talking about his recent article in the New Yorker, and gave this overview of a ‘taxonomy of reason-giving’:

[Tilly says] … We employ four kinds of explanations, he says: conventions (social formulae), stories (common sense narratives), codes (legal formulae) and technical accounts (specialized stories). And we get into trouble when we use one kind of reason in a context where another is necessary….

Tilly argues that we make two common errors when it comes to understanding reasons. The first is to assume that some kinds of reasons are always better than others—that there is a hierarchy of reasons, with conventions (the least sophisticated) at the bottom and technical accounts at the top. That’s wrong, Tilly says: each type of reason has its own role.

It’s not me who was smart enough to connect this to focus groups, but a Gladwell reader, Jason Oke, who wrote to Gladwell:

I think, as I gather you do, that how we feel about a brand, and which products and services we choose, is usually explained by a fantastically complex set of factors: the brands our parents used, the brands we see people around us use, the image of the brand, our personal experience with it, a sale, a half-remembered ad from 10 years ago, and so on. This is probably best explained as a story – we may both buy Tide, but there’s a different narrative that brought each of us to pick it up.

But in market research, the answers people give sound more like conventions: “It’s a good value”, “my family likes it”, “it tastes good.” And it seems that because of the artificiality of the situation, the perils of introspection, etc, most market research actually encourages people to answer in conventions, and doesn’t encourage the telling of stories. Many of these stories are probably complex and deeply buried such that they are hard to consciously access anyway.

I got into a bit of trouble with some readers when we were talking about Ugly Design and I suggested that it wasn’t necessarily a good idea to wholesale trust or believe what users are saying. Not that I think for a moment that I always know better than users (although, people pay me for my professional opinion for a reason). User involvement is always beneficial. But you need to know what to ask users, and you need to know how to ask it, and you need to know why you’re asking it.

In the very early stages of a project when you’re still in the exploratory/requirements gathering stage, what and why are less important. You’re looking for bulk and trends and a general insight into what the *real* problems are and what you might need to consider in order to approach it. (Users can be very helpful in uncovering the real problems, as opposed to the problems identified in your brief).

In this phase of the project, more is more. Spend as much time with your users as possible, and get as much as user input as you possibly can. (Here’s when working on an ‘internal’ project is a lot easier than a ‘public’ site).

Do you spend time with users in groups at this stage? Well, sure. You can. But there are two guidelines:

  1. make sure you also spend time one on one with users
  2. don’t accept all information at face value. Try to understand *why* people are saying what they’re saying. Try to understand how their presence in this group might colour they way they’re responding.

Sometimes you learn more from the dynamics of a group workshop and the codes that people use than you do from the actual words spoken.

(Documenting these codes and dynamics is a lot more difficult, and justifying decisions made on the basis of them is quite tricky, which is why you need to try to validate your learnings within groups with individual users).

As Jason Oke noted above, story telling is also a really powerful mode that can be used to get beyond the conventions and codes that users often utilise when working with you to identify requirements for a site or system. You should *never* accept that ‘the site needs to do X, Y, Z’ the basis that the user says so. (Even if that user is your boss/client/person who pays signs your paycheck).

You *should* look to users to provide evidence that a particular requirement is justified, and to propose possible solutions for that requirement. A great way to do that is through storytelling.

Storytelling is an excellent tool for designing system that use technology. People have funny relationships to technology. Many assume that their viewpoint is worthless because they are not confident with technology or they don’t have the technical vocabulary to express themselves with authority. Others work on the principle that if they use lots of fancy terms that they’ve picked up from other authorative technical people, their point of view will come across as more legitimate and we won’t realise that their range of knowledge about a system is actually quite limited, or that the system causes them troubles.

By raising up the under-rated mode of storytelling to above ‘conventions’ and ‘codes’ and ‘technical accounts’, we balance out the playing field and allow each user to communicate on their own, better transmitting their own personal knowledge and experience of the system (or the problems that the system is trying to solve).

Storytelling also implies that each account is an individual’s personal and therefore subjective account. This takes the weight of accurate self-reporting away from the user (and likewise the expectation from the researcher). As we know, individuals are hopeless at self reporting behaviour (so much that we do, especially repetitively, is done at such a subconscious level that we have little or no recollection of what *actually* happened).

From my experience, and from Jason Oke’s account, it seems that requirement gathering often places far too much faith in the ability of the user to articulate ‘what they need’ and ‘why they do things’.

This isn’t saying that users are incapable and shouldn’t be consulted, rather that there is an intermediary step where the person faciliating the process needs to be aware of the codes and performance factors that can influence the way that a user reports. This too is valuable information, and we should be more active in making use of it to better understand both our client’s requirements and their feedback.

So… tell me some stories about your experience with requirements gathering, focus groups, group talk and performance!

Technorati Tags: , , ,

  1. Really love this entry. You’re coming close to articulating an account of ethnography that’s applicable to all sorts of different research, not just usability testing. I really like the point that using a storytelling metaphor takes the responsibility off the shoulders of the user and shares it with the researcher and listener.

Comments are closed.