On documentation (or lack thereof)

I was talking to someone recently about doing some work with them. They said ‘can you send me some examples of documentation you’ve done lately’ – they wanted this to check that I could do what I said I could. Fair enough. Except, aside from the fact that all the documentation I’ve done lately is commercially confidential, so I’d have to hack it around a little to be able to show it to someone else… it made me realise how long it’s been since I’ve actually done the kind of ‘finished’ documentation I used to spend a lot of my time doing.

I just don’t work that way anymore, it seems. Sure, I still do wireframes every now and then, but never a ‘complete set’ and often with no where near the detail I used to include. Why? I think there are three reasons. Firstly, I tend to work on more of a strategic level than a detail ‘exactly where does that button go’ level these days. Secondly, I tend to work on projects where there is no time for that level of detail. And finally – and most interesting I think – I tend to work closer to the production team these days – more often are graphic designers designing and/or developers developing the very same stuff I’m wireframing at the same time. Investing too much in the documentation is a waste of everyones time – much better to do just enough to get them going and then work collaboratively with the team to do the fine tuning.

Personally, I think I should have been working more like this since forever.

Does any of this sound familiar?

Guerrilla Research – Recruitment

It’s been almost a year now that I’ve been doing predominantly ‘guerrilla’ design research. For me, this means testing in the field with a minimum of time, budget and fuss so that this kind of activity and the insight it provides is available to pretty much any client/budget/timeframe.

One of the first challenges for guerrilla design research is recruitment – the particular objective being to avoid using recruitment companies in order to reduce the cost of the project and also avoid the delay (often up to 2 weeks!) that is typically associated with recruitment.

A common approach to guerrilla recruitment is simply to rock up at a venue where your target audience is likely to congregate and to try to recruit on the spot. Typical venues might be a local Starbucks or a conference.

This is not an approach that I tend to use, for a couple of reasons – primarily because I really have to relinquish a lot of control over who I involve in my research… more than I feel comfortable with, given the responsibility that I have to my clients to provide them with useful insight. Also, sometimes the recruit briefs I need to meet are quite complex and require me to be quite selective when identifying participants. This is quite difficult to do on the fly and face to face… it can lead to either some awkward moments or spending time doing research with people who aren’t really quite right.

The approach that I have been using (and many of you are probably very aware of this!) is to use my online social network using tools such as my blog, Twitter, LinkedIn, and FaceBook. I have been pleasantly surprised by how successful this has been – on a number of levels.

Essentially how it works is that I work with my client to define a ‘call for participation’ that will be posted – the objective here is to get people interested in the research, to get more rather than less people to contact us, but hopefully mostly people who are at least close to right for the profile. I have to say, I am quite happy not to have to put together screeners any more (or review screeners received from recruitment companies). In fact… if I never saw another screener again I’d be perfectly happy :)

I do, up to a point, think about what ‘channels’ to use. As a gross generalisation, somewhere like Facebook is better for a less technical participant, where as Twitter is better for a more technically savvy participant… this is a gross generalisation though because more often than not, the people who I end up meeting are not directly in my network, but friends, family, colleagues of people in my network. This is the real power of ‘guerrilla’ recruiting, and a big reason why the ‘channels’ matter much less than having great people in your networks :) (Preferably great people with lots of friends, family and colleagues!)

To date, my experience has been that undertaking recruitment in this way has been cheaper (obviously – I don’t charge myself or my client ‘recruitment’ fees per participant), and faster (rather than weeks, recruits are sometimes filled within hours, definitely days of posting a call for participation). The most pleasant discovery has been that the quality of research participants is significantly higher than I had experienced when using recruitment companies.

In the past 12 months I’ve interviewed dozens of people and only had one ‘no show’ – and that was just a mix up with the dates as opposed to someone who just decided not to show. If you’ve done much research you know that it is not uncommon to have a day of six interviews lined up and only to get four participants to show up.

Some of the recruits I’ve had to fill have been fairly simple, but there have also been some incredibly complex briefs that I have no idea how I would have managed to effectively communicate to a recruitment company.

And the people who do turn up are fabulous – I’ve been amazed at how close to the ‘brief’ almost everyone I’ve met has been. They’ve been interested, enthusiastic, articulate and certainly not ‘professional research participants’ – the bane of any researchers existence!

Of course, you can always just take your design and show it to people in the office, take it home to your own family, show your friends – this can be very valuable. If you’re looking to do some research that is slightly more formalised, then perhaps you could consider using your own online social networks for this purpose.

It has certainly made me value even more the networks that I have in place and thankful for the great people who I interact with in these spaces.

(and, while I’m at it – if you’ve helped me with recruiting in the past year or so – thank you, thank you very much!)

Client-Centred Research

Word on the street is that the benefit of User Centred Design is hard to prove… I’m a fan of UCD as a process and I think that it is difficult to measure where process has been central to outcome, but today that’s a tangent. Part of the argument that is made is that a good designer can do almost as much by applying their design experience and expertise as what can be achieved through a UCD approach. In many cases, I heartily agree. Sometimes the process is more about the client than it is about the end users.

A project I’ve been working on recently is a case in point. This project is a pretty big deal – a lot of time and effort and, of course, money, have been ploughed into it already and there is plenty more to follow. Stakeholders have already been working on this project for ages when I get the call to come in and do my thing.

From a quick look at the proposed designs it is clear that there are some fairly significant issues that need to be resolved – both at the proposition level and at a more tactical, executional level. I don’t need research to tell me that and, because of my experience, I have a pretty good idea of what we need to do to fix this.

There is no WAY that I am going to be able to persuade all of the stakeholders who have invested so much in their current approach that they need to make the changes I’m suggesting… after all – I’m just one voice. One subjective, lonely voice… with my opinion alone, I have a long, lonely and difficult path ahead to try to get my clients to make the right decisions. Chances are, I’ll not be successful.

Cue user research. By conducting a quick round of user research I develop astounding powers of persuasion, because it is no longer my subjective, individual opinion that I am asking people to trust. Through the access I have to data accumulated in the course of research, I am able to validate my opinion with something much more substantial – the voices of the end users, the stories of my interactions with those end users, anecdotes, examples, tangible stuff.

Is this just a waste of project resource? Should I spend more time on trying to be more persuasive and less time on research? I think not.

In my experience, by talking to and observing your end users, by looking for patterns and themes, there is always something interesting to learn – almost always something that is directly applicable to your project, and if not, then certainly something that will help you develop in your ability to understand and design good user experience. If you’re interested in social interactions online, then staying in touch with how people are talking about, thinking about, using different tools is something that you should regularly be doing. Take this opportunity to throw in a few questions related to ongoing areas of interest.

At the end of the day – our job is to try to make sure that good decisions are made about the user experience of projects we are working on. If it were up to us alone, perhaps we wouldn’t need research. For as long as persuasion (of our clients, stakeholders etc) is a critical part of our role, research is an incredibly useful tool.

Are you using it?

Why collaborative research analysis rocks out.

dConstruct workshop - affinity sorting

(a quick definition, given that I’ve discovered that English is at least three separate languages: to rock out = to perform exceptionally well and give great satisfaction, as say, a rock band might ‘rock out’ on stage’.)

These days when I’m doing any kind of user research, rather than going to my secret consultant place and doing that consultant magic that results in a presentation of research findings, I much prefer to get into a big room with clean walls and several hundred sticky notes and my clients/project team, and to work out the research findings collaboratively.

Am I just being lazy and getting my clients to do my work? Well kind of…. but with good reason!

Why do it? Well, there are a few reasons.

Firstly, to combat what I think is probably the single most frustrating outcome of a research project – having your results either not accepted or immediately shelved, meaning that all of your work has come to pretty much nothing. By involving your clients in the process, they have a stake in defining exactly what the findings are, what is important, what is not. When you’re presenting the findings, you (or even better, the project team) are presenting the *team* findings, not just your own.

Secondly, to educate your client. To help them to understand that there is actually a rigorous process that occurs between the interviews or focus groups or whatever your research activity is, and when the findings magically appear in the presentation. To allow them to use the tools themselves when it is appropriate.

Thirdly, to get better results. Having your client with you will ensure that you apply appropriate rigor in reviewing research data. Not to say you don’t do this by yourself as well but it’s great to have the extra incentive.

Not only that but in this situation, three or four or five heads definitely are better than one. Take for example this study that Jared Spool shares in his article on the KJ Technique (which I’m referencing all the time!)

Back in the late 1970’s, the US government commissioned a study to look at effective group decision making. In the study, they asked 30 military experts to study intelligence data and try to construct the enemy’s troop movements.

Each expert analyzed the data and compiled a report. The commission then “scored” each report on how well it reported the actual troop movements. They found that the average military expert only got 7 out of a 100 elements correct.

Each expert then reviewed all of the other experts’ reports and rewrote their initial assessment. The average accuracy for these revised reports was 79 out of a 100.

What was different between the first report and the second? The experts didn’t have any new information. All they had were the perspectives of the other experts. When they added those perspectives to their own, their accuracy increased ten-fold.

It’s been my experience that if you can get your project team members (and their associated and diverse expertise) involved in the research analysis process, then you will most definitely get more accurate and more useful research findings.

So, how do you do it?

I’m sure there are a whole bunch of ways to do collaborative research analysis but I’ve gotten the most success from the following approach.

Firstly – encourage as many team members as possible to observe the research (if possible). Give them sticky notes and markers, give them the rules for writing sticky notes (one concept per sticky, clear handwriting in capital letters) and ideas about what kind of things should go onto the sticky notes. Don’t worry about the fact that you’ll have duplicates. Get them to write as many stickies as they can.

Then, when it comes time for analysis, you want a big room with lots of clean wall space. Plaster the walls with white or brown paper (whatever is easiest to get hold of) so you can move the stickies around en masse with ease. Then it’s time to get stuck into the process.

  1. start by defining the research question(s) – you should have done this before you undertook the research so this should just be a refresher. I like to get them written up and positioned somewhere highly visible in the room. This is what we’re trying to discover, the questions we’re trying to answer. They help maintain our focus.
  2. do a large scale affinity sort (follow steps 4, 5 and 6 from the KJ Technique). I know that this process looks completely chaotic at first… it is. Trust the process though, it actually does work. What happens is that you end up with lots of big groups with very vague names and some duplicates around the room. After you’ve done the very first sort, pick a big group and start dissecting it – look for groups within groups, and make sure that the group labels are actually meaningful in relation to your research questions. This is the tough part – you need to keep driving the group to keep seeking themes and meanings within the groups… and to sort and re-sort, and have lots of long, pedantic discussions – until finally the room full of stickies is completely sorted. (You can deal with the duplicate issue now by sticking duplicates one on top of the other so that they are not over-represented within groups).
  3. prioritise your findings. As a group – review all of the findings that you’ve come up with (each group is now a ‘finding’), and start grouping your groups together based on their relevance to your research question. You might have meta-group headings that are something like ‘Interesting but out of scope’ and ‘In Scope – High Priority’, ‘In Scope – Low Priority’ etc.
  4. then finally, go back to your research questions and work out what you’ve found. Based on the research you’ve done in this project, what are the answers to your questions?

Be sure to photograph all of your work, and then instead of the dreaded task of writing a ‘research report’, your job is then to gather all of this information into a digestible format for the team to use going forward.

And, of course, because they’ve actually been involved in the process, they’re much more likely to actually use it. Yay!

What do you reckon? Have you tried working like this? How did it go? Any other techniques that you’ve found work well? I’d be interested to hear what you think! :)