I found the design process utterly transformed once I decided to stop trying to be the expert and start trying to encourage a culture of experimentation.
Battles that would rage, angrily, for months – dying down when the provocateur was busy with other work but rising up as soon as they had a little time on their hands – these battles began to go away. Long frustrating and unproductive sessions of trying to explain, defend, rationalise why the design that I suggested had more merit than the many and varied suggestions (or requirements) coming from stakeholders all but disappeared. People who would usually sneer derisively at the design team became participating members of the design process.
It’s not perfect, it’s no silver bullet, but for me, it’s been a transformation.
And it’s pretty simple. To embrace experimentation you just need to stop talking about design in a Socratic way (other related but less civilised methods are also very common) and start formalising hypotheses and tests.
Stop having meetings to argue about which design approach is better - endless meetings with stakeholders full of defensiveness and crazy arguments where the people who tend to win are those who are loudest, most persistent or highest paid. Start making decisions based on lightweight research that provides evidence (sometimes stories, sometimes numbers) to support the design that most strongly supports the agreed goals.
Goals. That’s one pre-requisite you need for this experimental approach to work. You need to have agreed what your goals are for the design. What success looks like. Without this agreement, no change to methodology will save you.
The experimental mindset is an egalitarian approach to design. It allows that anyone can suggest a design solutions and, rather than argue endlessly about whether it is better or worse than other approaches, you design a test. Find out how to find out which design works best.
Hypothesis, prototype, test.
There are loads of tools you can use to test ideas quickly – from some quick in person user research, to some A/B testing (if you’re not set up to do A/B testing, meet your friend Google Content Experiments and get onto this immediately), to an online card sort, to one of the range of tests that places like VerifyApp offer. The methods for testing are limited only by your creativity and are mostly inexpensive.
Sure, you can’t design from the ground up this way – you will still need a good designer that you trust get you to a good starting point from which you can experiment up, but once you’ve got the framework in place, don’t waste time and goodwill bickering about the details but encourage experimentation throughout the entire organisation. You’ll raise the overall ‘design IQ’ and happiness quotient of your company, your design team and, most probably, even yourself.
I’m sure that many of you will have heard about the very worthy Alpha.gov.uk project, the first prototype of which was released earlier this month.
If you’re a user experience practitioner, this should particularly interesting to you.
By way of a quick background, the AlphaGov project was formed in response to findings from a report by Martha Lane Fox, Revolution not Evolution into Government online services and opportunities to improve. (As a tangent, I’d love to see her in a cagefight with Lou ‘The redesign must die‘ Rosenfeld)
In this report she recommended the introduction of
“a service culture, putting the needs of citizens ahead of those of departments”
To test, in public, a prototype of a new, single UK Government website.
To design & build a UK Government website using open, agile, multi-disciplinary product development techniques and technologies, shaped by an obsession with meeting user needs.
See. It doesn’t get more UX-interesting than that right? It reminds me quite a bit of the D7UX project I worked on with Mark Boulton and the Drupal community, so I’ve been following it’s progress with a keen interest.
Now, go have a play with the prototype and see what you think. I’m actually not going to comment on the UX of the prototype today, partly because it’s actually quite difficult to do so. I’ll get to that later.
What I want to talk about today is the responsibility that playing out a project like this in public brings with it and how, in my opinion, AlphaGov have let down both the UX and Inclusive Design/Accessibility professional communities.
What you do, not what you say
Let me start by saying that I have a lot of admiration for the the ambition of this project. There is a lot that is good about it. There are also a lot of smart and talented people on the team.
The first thing that strikes me as strange is that on a project that claims to have an obsession with meeting user needs, the team contains a visual designer and a content strategist, a general strategist and multiple search analysists but NOT a user experience lead.
That’s right. We have an obsession with meeting user needs but not so much that we’ll actually hire someone who has extensive experience in actually making that happen.
Now, the project was fortunate in that it had Richard Pope, who I first met when he was a very UX-savvy developer at Moo and who played the Product Lead role on AlphaGov. As far as UX resources go, you could do a lot worse.
The team also recruited Paul Annett later into the project. Paul also has some UX experience but, as I understand it, his role was primarily as visual designer, making the interface a nicer place to be.
Without commenting on the interface itself, the lack of a rigorous approach to user experience is very evident in the way that the team talk about the work that they have done and their ‘design rules‘.
We spent the first two weeks in February recruiting a team from inside and outside of government, talking through the scope, agreeing some design rules and agreeing a vision for the Alphagov product based around the recommendations of Martha’s report.
We ended those two weeks with a list of prioritised user needs (based around search analytics from Directgov, Hitwise and departments),
We roughly grouped into functional areas and stuck to the wall. Each card (or user story) represented a user need, prioritised roughly from left to right and top to bottom.
Crucially also there was a fair amount of @tomskitomski and @memespring‘s product experience. All this was more than good enough to get on with twelve weekly development sprints.
More than good enough, eh? For many projects this would have been more than they ever had to work with.
But this is not just any project. This is a groundbreaking, whole of government initiative that claims to take a User Centred approach and be obsessed with knowing and supporting the end user need.
I think a project like that needs to demonstrate User Centred-ness a little more rigorously. For example.
Who is the audience?
At no point that I saw did the AlphaGov team ever apparently think deeply about what kind of an end user they were going to prioritise. They talk about ‘thinking about who our users were’ and having a ‘user-base of all the entire adult population of a country’.
As User Experience practitioners we know that although you might want the whole country to use whatever you’re designing, you need to put a ring around the kind of users you MOST want to support.
As designers we always privilege some behavioural attributes over others, even if we don’t articulate it. By not thoughtfully articulating this, you risk either an incoherent approach to the experience design or resort to self-referential design (designing for your own behaviour – something that is incredibly difficult to overcome).
You can’t take a User Centred approach to design when your user is ‘Everyone’. You need to define who your users are. You must clearly identify the behavioural characteristics that you most want to support and focus on designing to best support these.
There are many valid design approaches that do not require such a clearly articulated definition of the end user, but these are NOT user centred approaches. Thinking generally about ‘users’ while we design is not doing user centred design. I think it’s pretty irresponsible to suggest that it is.
AlphaGov sends a message that you can say you’re doing User Centred Design but you don’t have to show any evidence of a UCD process – audience definition, research, user involvement, design principles that actually track to specific behaviour attributes.
For example, it would have been great to see some personas developed and shared for this project – even hypothetical ones that drew on the data/insight available to the team. As well as helping the team avoid the problem of the ‘elastic user’ (particularly problematic when you do think your target audience is everyone), it would also help us be better able to evaluate what is good and bad about the prototype. It would also have demonstrated that the team was actually practicing User Centred Design.
(Elastic user, for those not familiar with the term, was coined by Alan Cooper to describe the way that while making product decisions different stakeholders may define the ‘user’ according to their convenience, often resulting in self-referential rather than user-centred design. More here).
This leads us to one of the complexities of the AlphaGov audience which is that, in reality, rather then being obsessively user-centred, the project had two competing audiences. The largely undefined end user and, often more importantly, the stakeholders who would ultimately decide the fate of the project – public servants. These two audiences have VERY different motivations and goals for this project, and the impact of the latter on design decision making was never clearer than when the accessibility topic came up.
On Accessibility and a conflict of interest
Again, from what I know, there was no formal expert accessibility (or inclusive design as I prefer to call it) consultancy on the project team. This is not to say that the team didn’t have quite a bit of knowledge about the mechanics of accessibility (the impact of technical decisions on whether something could be certified ‘accessible’).
Accessibility should start with research and consideration, not with box-ticking or sprinkling a few standard accessibility features – especially in a government context where a user journey regularly extends into the real world (Booking a driving test? You’ll probably want to know the facilities at the test-centre).
Actually, what I picked up from discussions about this on Twitter and elsewhere was that actually, it was the other target audience – the stakeholders – who most influenced this decision. If they put the focus on accessibility, they’d have to take away some of the ‘shiny’ – AlphaGov would end up feeling like Just Another Government Website. Rightly or wrongly, the shiny would help excite the public servants to approve and fund a beta version of the prototype.
Perhaps it was a noble sacrifice… who knows. Point is, it’s far from transparent.
The message that the world takes away from this exchange is that accessibility, even when your audience ‘entire adult population of a country’ is optional. And that accessibility can be ‘done later’ not, as they had first set out, built into design considerations from the outset.
These are really bad messages to be sending and, given how publicly visible and lauded this project is, sets the work of many amazing inclusive design specialists back considerably.
It’s hard enough to sell in good accessibility work already. AlphaGov just made it harder.
Activity Based Design and Search Analytics
OK. So I will talk briefly about the prototype… but mostly to discuss how the data you have access to can significantly shape your design.
The team have published very little information on the data that has guided their design decision making for this project but we do know that search activity has influenced it heavily. A team of sixteen people included no UX lead (sorry, did I mention that already?) but two people doing search analysis.
In the design rationale blog post, Richard Pope implies that search logs strongly influenced the design and information architecture strategy for the prototype.
we spent the first couple of weeks scouring search logs and analytics for the various central government websites; thinking about who our users were and generally discussing the kind of thing we were setting out to make
Based on what we learned from looking at search-logs, we knew that there was a relatively small subset of tasks that require the majority of people need to interact with government online. So we should do less and focus on tasks.
Since for the vast majority of people their web journeys (finding out the date of the next bank-holiday, or reporting a lost passport) start with a search engine rather than a direct visit we should think of Google as the homepage and we should also feed Google, Bing and other search engines nice friendly urls.
If someone is just landing at a page on your site then it’s helpful to start thinking of every visit being a new user, assuming they have no prior knowledge of the structure or content website they have landed at.
It is really difficult to evaluate this prototype from a user experience perspective, given the competing target audiences. The best you can do is try to recall the last few times you interacted with a government website and try to reenact that here. Every time I do that I come away with a lingering feeling of disengagement. There’s something that search logs probably don’t really show which is that this is MY government. For better or worse, I have a long term and multifaceted relationship with this government and yet, every time I encounter this site it (by design) makes me feel as though this is my first visit. I find that really unsatisfying and kind of perturbing.
Now, this is not a professional design critique, this is a qualitative research data point of one. But it’s not something that you’ll ever pick up from search stats and analytics. I could bore you further with how I find the promise of localisation with this infinite noob status even more perplexing, but you’d have to spend time with me to understand it. And then spend some time with a bunch of other people to see if this is a common theme or just me being an edgecase.
And people will never post this kind of feedback on GetSatisfaction. (Most people can’t really articulate this kind of weird feeling and wouldn’t think that it was important enough to comment on compared to, say, a bug. You need a good facilitator to extract this kind of data).
To do really good user experience design you need a mix of data points. If you privilege one set of data, you’ll see that in your design. I think we’ve got some of that going on with AlphaGov.
Quantitative data is fantastic. I’d love to see more of what the team had to work with and how they applied it in their design process. But it’s just one kind of input. Qualitative research helps you better understand your end users and thereby to design better for them.
People who do User Centred Design do Qualitative Research.
User Experience is a Time Soak/Non-Agile
Which leads me to the final problematic sub-text that I felt emanating from the AlphaGov team which was essentially that ‘we’re as user centred/accessible as we can be given that we only have 10 weeks to build this thing’. This perpetuates the myth that User Experience can be a time soak, that it slows you down, that it doesn’t really have a place in an Agile methodology.
This is where having an experience UX practitioner on the team from early on could have been helpful.
It is certainly true that historically, Agile and UX have had a fairly vexed relationship but these days many practitioners are experienced and adept at including both user research and ux design into the most demanding agile iterations. We have a toolkit of lightweight qualitative research approaches that work beautifully in this kind of fast paced and responsive environment. UX does not have to be a laggard either at the outset or in the throes of an agile project.
The ten week project timeframe is absolutely no reason to not include real practice of user experience in the process. You may need to find someone who has experience working this way (not all UXers find this kind of project much fun), and you may need to be creative in the way you structure the work, but you should definitely be doing it. Particularly if you’re setting an example of how projects should be done, as the AlphaGov team certainly was.
I want to repeat again, this is a very worthy project and many of their design principles are, I think, sound. For many commercial projects, the methodology that they’ve applied and shared is absolutely appropriate. But the bar is set higher here.
By doing this project in public, by making such a big deal of putting the end user needs and their importance to the project, the AlphaGov team have set themselves up as rolemodels. They’re sending messages about the the way things should be done. They’ve made quite a rod for their back.
If I was just a member of the community, I’d probably be nothing short of delighted with what they’ve achieved. Unfortunately, as a User Experience practitioner, I feel kind of glum. This project has talked the talk of caring about the end user, of placing their needs at the centre of the project and above the needs and desires of government, but at every step, they’ve done little to set a good example for how others should actually do this.
I hope AlphaGov does move forward into BetaGov.
But I hope, if they do, they take a moment to think about what the public performance of AlphaGov and then BetaGov means for their professional community.
Either stop calling the project User Centred, or hire someone to really focus on user experience and do more to share how they’ve integrated real user insight into their design strategy and implementation.
There’s a big opportunity to set a good example to a big audience here. Let’s take advantage of that opportunity and show the UK Government, digital industry, hell, the whole world what projects really look like when they’re user centred, – that they don’t have to be cumbersome, expensive and slow.
Imagine that, a properly user centred government website that was agile, and shiny and amazing. Now, that’s something to get excited about.
I had the pleasure of mentoring at the first Design Jam in London today. The event brought together about 50 UX designers from student to seasoned professional to form teams of about 4-5 to design a solution in response to a design challenge.
The challenge for today was:
What is the ideal interface to keep track of previously viewed online content, across multiple devices and locations?
You can see what the teams came up with by checking out each of the team wiki pages.
It was a lot of fun running around from one team to the next seeing what they were working on and, hopefully, helping to guide them towards a solution to present at the end of the day. It was really interesting to be able to observe nine teams approaching the same design question, and to see where the common challenges emerged. Some observations and advice:
Spend less time choosing your idea and more time defining it. Specifically, what problem are you solving?
Peter Drucker, a business management guru said ‘Ideas are cheap and abundant; what is of value is the effective placement of those ideas into situations that develop into action.’ Nowhere is this truer than at DesignJam. If you want to have something interesting to present at the end of the day, you need to quickly identify a specific problem that you can solve, and then you need to be able to describe that problem in a concrete story. ‘ Keeping track of previously viewed online content, across multiple devices and locations‘ is so broad as to be meaningless from a designer’s perspective. But, being able to re-find a hotel website I saw a week ago when considering a holiday, or the location of the event I’m going to tomorrow, or finding the link to that funny website my friend emailed me about the other day – those a real, concrete, solvable problems.
It doesn’t really matter which one of these you choose, what matters is that you quickly identify a relatively small, concrete problem that you can solve and that you can describe the problem clearly and believe that the problem is real, and describe how life will be better for people with this problem resolved.
The elevator pitch technique is one method you might want to consider to help get yourself to a stage where you *really* *clearly* understand what you’re working on and why.
I really can’t stress how important this part of the project is – this is the foundation on which all the rest of your work is built on, and the most important thing is not *which* idea you choose, it’s about how clearly you’ve defined the problem you’re going to solve and the value you’re going to deliver – your value proposition.
Define your audience by understanding the important behavioural characteristics.
Ah, the vexed issue of personas.I saw a lot of personas at DesignJam today and very little evidence of them being used as part of either the problem or solution definition. Personas *can* be very valuable but only if they’re used in the right way and that is as a tool to help you understand what are the behavioural differences that are significant to your design problem, preferably informed by real data points (your mum, husband, grandfather do count as data points in a DesignJam scenario!).
Time is precious in a DesignJam environment (as it is on all the project we work on, right?) – we need to make sure our time is being spent in the best possible way. I witnessed too much time being spent making personas because it felt like the next logical step in the design process. In most cases, I would have preferred to have seen groups spend time defining usage stories or tasks and then, if it became clear that there were divergent behaviours and we needed to choose to support one kind of behaviour or another, then capture that somehow – and perhaps a persona is a good way to make that behaviour more understandable.
Having said that, one of my favourite designs today emerged in response to an ‘extreme’/edge case persona – so persona’s can be a starting point – but what drove this design was not the persona as such but the behaviours we were able to identify that were specific to that persona (and very different from our own) – in this instance, the use of links in email as a primary trigger point for viewing websites, also getting relatively few emails from relatively few senders.
If you must do personas, then do as few as possible. If you’ve got more than three personas, I want to know why.
If you’re going to spend time making personas, then I want to see you actually using them in your design process.
Get sketching! Generate and evaluate lots of design solutions before you start wireframing
So, all that time you probably spent trying to come up with A Good Idea, spend it here instead. Quickly generate as many ideas as you possibly can. I reckon it was at least 2pm before I saw people starting to sketch out ideas at DesignJam today (teams started tackling the design problem at 10am and were supposed to present at 4pm).
A really popular approach to generating lots of ideas at the moment is to do 6-up wireframes another technique I quite like is Design Consequences. However you do it, the key is to get as many ideas as you can onto paper. And then – once you’re out of ideas – to use your clearly defined design problem and whatever user behaviours or personas you have defined to evaluate which aspects of which ideas are strongest.
Once you’ve evaluated the first round of ideas and you’ve got fresh ideas in your head – do another round of visual brainstorming. Rinse, repeat until the answer becomes obvious. Eventually, it will. Then everything will start falling into place.
A group is a resource and a liability (user your numbers, appoint a facilitator)
When you’re designing with a bunch of other designers (or actually, with any group at all), there are two key things to remembers – firstly – use all the people in your team, get them all actively designing, make sure everyone is sketching and contributing ideas, remember to do things quietly and individually sometimes and to do things collaboratively and together at other times.
Secondly – make sure that someone is driving the team – keeping you on a schedule, working out how you’re going to get from here to the end of the project, making sure that you’re staying true to the project problem you’ve defined, making use of the personas you’ve defined, keeping everyone focussed, on track, and working productively. Have this discussion at the beginning of the project rather than waiting for a ‘natural leader’ to emerge (especially if you’re working somewhere where politeness is at a premium and potential leaders might be nervous of treading on other team members toes)
Pitch clearly and persuasivelyThe day wraps up with each team presenting their design to the larger group – for me, this is as important as all the design work you’ve done throughout the day. A clear, focussed and compelling presentation enables you to convey to the group what you’ve been working on, what problem you’re solving, who you’re solving it for, and finally, to show the design solution you’ve come up with.That clear value proposition and the user stories or tasks that you’ve defined come in handy yet again and show be key to framing your work in a way that is understandable and compelling to your audience.
Don’t think of this as ‘just the presentation’ – as much as any of the design work you’ve done throughout the day is great experience and practice for your day to day design work, the same couldn’t be truer for this part of the process. As designers, we’re only ever as good as the design we can convince our client/team to implement and this means that we’re constantly presenting our work – explaining what the problem is, why we’ve done what we’ve done. This is something that, as designers, we should be able to do at the drop of a hat because of the preparatory work we’ve done earlier in the design process.
While these thoughts are specifically in response to the DesignJam day, I think they’re pretty much universally true to any design project and very common issues that come up on projects I’m involved with. The hothouse environment of DesignJam brought it home, yet again, how difficult it can be to facilitate a team around designing a solution – it’s tough work but very rewarding.
Over the past few weeks I’ve been wrapping up and kicking off a bunch of projects. It is during both of these phases that I am reminded how incredibly valuable it is for me, as a UX practitioner, to proactively encourage my clients to clearly define the goal for their project and to create success criteria – ways that we can tell whether or not the project has been successful.
In my experience it is very often left up to me to make sure that the project goals are as clear as they need to be. Many clients come into projects wanting ‘to improve the usability of our website/application/etc’, in my experience that is most often too vague a project goal as to be useful.
The kind of project goals that are really useful go more like this:
We’re getting lots of traffic to the site but not many are joining/buying/contributing/coming back. We want to fix this.
We’re getting lots of calls from people who have visited our website but still don’t know what we do. We want to fix this.
People are coming to our site and doing X but we really want them to start doing Y, we want to find out why this is happening and what we need to do to address it.
As soon as you start defining more specific project goals you can immediately see the way that success criteria start to become immediately apparent.
Measuring success criteria
Some success criteria are immediately apparent and easy to measure, for example return visitors, increased membership, activity or sales. Ideally you want to put some numbers around what you’d consider would define this project as ‘successful’, but even just identifying the metrics that you will use to judge the success of the project is a good start.
Some success criteria are less easy to ‘measure’ but don’t let that discourage you. Often for these kinds of criteria I’ll use a round of research to determine whether or not we’ve been successful – those things that are difficult to quantify are often quite easy to examine using qualitative research. I find myself more and more using a last round of research to ‘check off’ the less quantifiable success criteria for projects.
What’s in it for me?
Failure is a built-in risk of success criteria, but don’t let that put you off. Failing knowledgeably is actually an incredibly useful learning experience for practitioner and client alike (and yes, I speak from experience). I would argue that by defining clear goals and success criteria you are going to do nothing but increase your chances of success in a project.
Clearly defined project goals allow you to make all kinds of good decisions on a project and it can impact everything from design decisions through to who you recruit to participate in design. Just the other day a project I’m working on managed to avoid wasting a number of research sessions on ‘people we felt we had to include’ because we were able to really clearly define why, for the purposes of achieving the goals for this project, they were not our target audience. Without the clear goal we would never have been able to come to this decision.
Clearly defined and measurable success criteria similarly guide you through decision making throughout the project, but they also continue to be useful well after a project has wrapped up. Of course, we use them to judge the success of the project, but we can also use them to communicate this success to others in the organisation and beyond. Clearly defined and measured success criteria give us something tangible to talk about and take much of the ‘touchy feely’ fuzziness out of User Experience and that makes it much easier for more people to understand and appreciate the value of the work we have done.
My name is Leisa Reichelt. I am the Head of User Research at the Government Digital Service in the Cabinet Office.
I lead a team of great researchers who work in agile, multidisciplinary digital teams to help continuously connect the people who design products with the people who will use them and support experimentation and ongoing learning in product design.
If you're interested in working with me or would like to talk more please email me