When experience matters (and when it doesn’t)

A little while ago I wrote this, which turned out to be more controversial than I intended.

I wrote it after having several encounters, in close succession, where one of these had happened:

  1. Recruiting scenarios where people assumed that previous domain experience was more important than the overall research experience, and capability, of the researcher.
  2. Working with a researcher who had domain experience on a particular topic who kept shutting down team conversations based on experience they’d had in previous projects, which the rest of the team had not been involved in.

These are both anti patterns of domain experience in user researcher and should be avoided.

Recruiting user researchers

When you’re recruiting for a user researcher, there are three things that teams should be looking for:

  1. Firstly, ensure that the user researcher has experience in the methods of user research you expect you’ll be using on the project. For example, don’t hire someone who has a decade of experience doing usability testing when you need someone who is strong in contextual research to work on the discovery phase of your project.
  2. Secondly, be confident that the user researcher cares about the quality of the service that their team ultimately delivers, will be able to hold their own in the team and will be effective in communicating research findings in a way that compels the team to act on them. This is the hardest criteria to meet. Too many user researchers have had a great time doing research and making reports, then leaving the teams to do with it what they will. You want a user researcher who wants to have ‘skin in the game’, and wants to see their research valued and used in making service design decisions and in setting priorities for the team.
  3. Finally, if the researcher has previous experience in the subject domain of the project, this can be an advantage. But only once the first two criteria have been met. These first two criteria are much more important for user researchers than subject matter domain expertise, and they’re also pretty hard to find.

When you hire a user researcher, the domain they should be most expert in is user research.

In teams, diversity is a strength

For most projects, having some subject matter expertise is essential. In most cases, that would be at best inefficient and at worst dangerous to do otherwise.

What is equally problematic, though, is a team full of people who have extensive experience working on the problem space they are just about to tackle.

Teams like this have so many shared mental models and assumptions about how things work, what things mean, where the constraints are, and how people think and work, and, despite their experience, not all of these things are right.

The CivicPatterns website calls this the Clean Room pattern and says:

If your project operates within any bureaucratic system, ensure that the person responsible for its design knows as little as possible about how the existing system works.

Most people hate dealing with bureaucracies. You have to jump through lots of seemingly pointless hoops, just for the sake of the system. But the more you’re exposed to it, the more sense it starts to make, and the harder it is to see things through a beginner’s eyes. Therefore, when building a system that helps someone bypass bureaucracy, start by designing how the system should be, with as little pre-knowledge as possible, and then, when you need to add any complexity, work as hard as you can to hide that from the user.

Lack of diversity in experience-levels (and lack of diversity in general) in the team will reduce their ability to consider a full range of service design options that can streamline the experience for users. This will limit the potential for transformation.

There are some roles where experience the domain of the project is essential and teams would be foolish not to include them. Designers and user researchers are not those roles.

Design your teams so they have diversity, in gender, in age, in backgrounds and in subject matter expertise and you’ll design better experiences for your users.

Death to ‘it depends’

Lately I find myself on a mission for mass simplification. Possibly over simplification, but I’m not sure it matters.

It’s one of the things I care most about at the moment – how can we simplify what we are asking people to do so that there is nothing else they can do but start doing it, instead of following their natural inclination to make a list, hire a consultant, write a white paper, do anything but doing the thing.

It requires that I stop saying (or even thinking) one of the things I have probably said most in my entire working life – ‘it depends’. That’s hard, but I think it’s the right thing to do.

It depends is paralysing.

It is a stop sign. It tells people they can’t do anything yet. That it is complicated.

And it presents things as  complicated at a time when the person asking the question usually doesn’t have the capability to understand they subtleties you are trying to explain.

Of course, most things are complex and it usually does depend (on context), but knowing what to do in the face of all of that is less important than just doing something now. Ideally something that gets you on the right track to having the ability to ask more nuanced questions, to have a more sophisticated understanding of the thing you’re doing and the context you’re doing it in.

What does this mean? For me, at the moment, this means trying to create a massively simplified message for teams across government about how they should start doing user research.

Lots of different teams, different projects, different audiences – a whole world of it depends.

Forget all of that. What’s something massively simple that all of them can start doing right now.

Here’s an idea.

  1. Start doing some research in every iteration (we’re all doing agile)
  2. Start by getting 5 or 6 people in a lab, give them the thing you’re making and a task and watch them use it. Talk to them about what they find difficult. Maybe also ask them about how they use the internet in real life, what kind of interaction they’ve had with government before. Don’t explain your thing.
  3. Get as many of your team as you can to observe (hence the lab). Aim for 2 hours every 6 weeks for everyone as a minimum.
  4. Capture what you hear and see on post it notes. Do affinity sorting to analyse research. Write down what you learned (your insights) and what you’re going to do about it. Do those things, come back and do the same in two weeks.

Now lets all start saying this to everyone, every chance we get.

It won’t be the best thing for every project to be doing all the time, but it is way better than doing nothing.

As soon as you start doing this, you will start to learn about different and better things that you can do (if for no other reason than because you’ll need a user researcher to do this and they can tell you. Before you start this, you probably won’t have that person).

Now we just need to do this for all the things…

Related: I watched this video from the ever grumpy but rather clever Russell Davies the other day. He’s scaring the pants of people who are studying to become Planners in the advertising industry and telling them that the best strategy is not the most creative one but the one that actually gets anyone to actually do something.

There’s an interesting section on the value (or not) of research as well.

Help Joy help you. On the unusability of internal systems.


I am out here for you. You don’t know what it’s like to be ME out here for YOU. It is an up-at-dawn, pride-swallowing siege that I will never fully tell you about, ok? Help me… help you. Help me, help you. – Jerry Macguire


This is Joy’s notebook.

At the airport earlier today I had to switch my ticket from one flight to another. Joy was the customer service person who helped me do this.

Joy’s notebook is about two inches thick, she’s created an A-Z index for it, it is packed full of handwritten notes about how to do different tasks in the various system she uses – steps that need following, codes that need inputting. It sits beside her every day, beautifully decorated (with stickers) evidence of the horrendous usability of the software she uses to get her job done.

Joy has been working for this airline for years. She told me that some of the stuff that is in her notebook she doesn’t need anymore – either because they’ve upgraded to a new system or that, after years, she’s finally managed to memorise it.

She told me that each time they upgrade the system it seems to get harder, not easier, to use. Joy told me that all the customer service reps have a notebook like this. You can’t use the systems without one. Joy is digitally literate and confident with the computer, but it is impossible to use without the notebook.

Joy is frontline staff for a major international airline. She is delightful and doing her best to do great work and look after her customers, but this is what she’s got to work with. I wonder if I’d be so cheerful if I was in her seat. No wonder so many others are not.

Internal systems are the tools we give our frontline staff, the people who are in charge of the customer experience for the face to face and telephone channels. If you pay attention to people using these systems (either out of general professional interest or because you’re fortunate enough to work with the on a project), you’ll find that notebooks like Joy’s are not uncommon. They’re everywhere.

They are everywhere because the people who bought or made the system didn’t even think about the experience for internal staff. The internal staff who are stuck using it are so far away from the people who bought that expensive crap that they’ll never know how awful it is (or their jobs are in peril so they don’t dare complain).

Internal systems allow an organisation to deliver great customer experience throughout the customer journey. These systems let people like Joy be fast (or not), accurate (or not), joined up to the rest of the organisation (or not).

And yet, all the time, we say ‘It does’t matter, we’ll sort that out with training’, ‘Call the tech writers, we’ll make a manual for this system’,  ‘Don’t worry, we’ll inflict this piece of crap on our employees, unlike our customers they’re stuck with us’. Except they’re not really, are they.

This might have worked with Joy but as our employees begin to join our workforces as digital natives, familiar with the well designed exteriors of organisations, how well do you think they’ll take to the tools we offer them to book the tickets, create the content, manage the accounts.

I kind of fancy that they’re going to start telling us to lift our game, to stick our crappy internal systems up our jumpers, and to give them so decent tools so that they can be as efficient as we want them to be. So that they can offer the kind of customer service everyone wants to experience.

I really hope they do. I hope that they will demand that we do allow time to work on the usability of the content management system, not just the website. That a company will be able to win on customer service because they actually bothered to hire design team that lets the customer service people offer faster, better service. That the news company that optimises the content management interface wins because their journalists can write more, better content rather than battle the content management system for hours a day (true story).

And I really hope that one day Joy will be able to stop battling the interfaces she uses to give such great customer service and leave her notebook at home.

If you’re going to do this user experience thing properly, you’ve got to look at all the angles. If you respect for your employees and your customers you need to care about the user experience of internal systems. Challenge yourself to solve the often more difficult design problems of internal systems, and know that by doing that, you’re creating a better user experience for all.

Strategy is being on message.

If you’re trying to implement strategy I reckon you need to spend a good 30% of your effort on communications. Probably more. You need to think more like someone who is trying to sell expensive trainers and less time making power points, spreadsheets and gantt charts, or doing whatever the thing is you’re officially paid to do.

This is the most important thing you can do to help, I’ve said to teams at GDS and people in other government departments just this week, the most important thing you can do to help is to say the same things that we say to anyone you get the chance to say them to. Of course, I am taking as a given that people will always do the best work they’re able to do in any situation, we need that too.

If you want to implement a strategy, do what the marketing people and the political advisors say – get on message and stay there.

This requires two things:

  • working out what your messages are (the fewer the better)
  • expressing those messages clearly (choose your words carefully)

Achieving both of these things takes time, effort and attention. It doesn’t tend to happen organically, without consideration.

Work out what your messages are

There might be lots of behaviours that you want to change, but you’ll only ever be able to get a couple of messages out to people – and only then if you say them over and over and over again.

Work out the one or two key messages that you can focus on – look for behaviour changes that are relatively easy to achieve (or that sound like they might be). Bonus points for finding behaviours that, when adopted, will naturally lead towards other behaviours you would like to see.

An example. Over the past year or so, we have tried to achieve a pretty significant change to the way that people in government use user research in digital projects. Mostly we wanted research to become a discipline that was as agile as the developers, product owners, designers and was able to be involved throughout the product lifecycle so that we could be more effective. We wanted to stop  people outsourcing a bit of research at the beginning and end of the project and complaining that it wasn’t very useful. We wanted user researchers to be able to be effective contributors to the success of the projects.

You have to do a lot of things differently to achieve that, but we’ve focussed on three main behaviour changes:

  1. project teams to do research more regularly throughout the project lifecycle
  2. researchers to be able to work in way that required less documentation and more light weight and regular communication with the team
  3. everyone in the team to participate more in the user research, for more people to actually see end users interacting with the service we’re making.

If we can achieve those three big things, then we can start to worry about the detail of exactly what happens in the team, who does what, when and how. Those will be good problems to have, and we’ll worry about having a clear position on those when we need to.

These are three very noble goals, but they don’t feel easy to implement and achieve.  So, we needed to look at our messaging.

Express your messages clearly

User research is a team sport poster

We’ve taken those three objectives and crunched them down into three soundbites that you hear more and more people starting to say around GDS and across government. They are:

  1. User Research is  team sport (often followed with, ‘we don’t do this for us (researchers) we do this for you (the team)). This doesn’t really tell you exactly what to do, but it tells you a lot about the new philosophy of user research. That it requires participation and it is open and collaborative.
  2. Do research in every sprint (often followed by, you need a user researcher in the team for at least 3 days every week). Making the call to action very specific is both motivating and rewarding. It helps teams know what they should be doing and for those who are doing it, confirms they’re on the right track.
  3. Everyone in the team should watch user research for at least 2 hours every 6 weeks. We make a lot of use of the UIE research around Exposure Hours. It continues the ‘team sport’ message, validates the need to do research each sprint, and is another clearly measurable goal.

We’ve learned that making your goals attainable is important, and that using specific measures (every sprint, 2hrs every 6wks) seems to be very powerful. This is a big change for people who are used to answering most question with the preface ‘it depends’.

By reducing our overall goals down into three simple soundbites it makes it easier for everyone in the user research team to say these things over and over again, but it also makes it easier for people who are less familiar with user research but who are integral in organising budgets, people, timeframes – the things that are critical to us being able to achieve this – to know what we want them to do and to ask for the same things without feeling they need research expertise.

Although we know that these three requests aren’t necessarily the best way to do research in every project, we think they are the best way to get an empowered researcher into a project – once we’ve achieved that, tweaking the program to better suit the project is relatively simple.

These are the words that will be adopted throughout the organisation and, as we know, words are powerful, so you should think carefully about the ones you choose.

This example that focusses on user research, but you can apply this to any area of your business or project where you are looking to get a strategy implemented.

Be on message every chance you get

Of course, once you work out your key messages, you then need to take every opportunity you get to repeat them. This is where ‘saying the same thing’ really kicks into action. Really, literally, say the same words. Use your soundbites.

After a while, you will start to feel a bit silly saying the same thing over and over again. Remind yourself that no one (except you) hears you say it every time, and use your key messages in conversations, presentations, blog posts… we’ve even made posters for ours and post them around GDS and government departments when we visit.

What success sounds like

One day, you’ll hear someone you’ve never spoken to say it back to you. Or even better, they’ll say it to a large crowd of important people in your organisation. You’ll only know that if you’ve carefully chosen your words.

When that happens, have a quiet celebration because, unlike many other strategies before it, yours is now making progress.


This is the seventh post in a series of rambles around the topic of strategy in the general vicinity of user experience which I’m posting as a kind of obituary to the book I almost finished writing then realised was pretty much completely wrong.

I’ve been writing this instead:

  1. Everyone is doing strategy right now
  2. Strategy doesn’t live in a silo (or there is no such thing as UX Strategy)
  3. Strategy fast and slow (or strategy is culture for breakfast, lunch and tea)
  4. Strategy is a team sport
  5. Good strategy is modular
  6. Why words matter (more on the relationship between culture and strategy)