Experimentation beats expertise

I found the design process utterly transformed once I decided to stop trying to be the expert and start trying to encourage a culture of experimentation.

Battles that would rage, angrily, for months – dying down when the provocateur was busy with other work but rising up as soon as they had a little time on their hands – these battles began to go away. Long frustrating and unproductive sessions of trying to explain, defend, rationalise why the design that I suggested had more merit than the many and varied suggestions (or requirements) coming from stakeholders all but disappeared. People who would usually sneer derisively at the design team became participating members of the design process.

It’s not perfect, it’s no silver bullet, but for me, it’s been a transformation.

And it’s pretty simple. To embrace experimentation you just need to stop talking about design in a Socratic way (other related but less civilised methods are also very common) and start formalising hypotheses and tests.

Stop having meetings to argue about which design approach is better  – endless meetings with stakeholders full of defensiveness and crazy arguments where the people who tend to win are those who are loudest, most persistent or highest paid. Start making decisions based on lightweight research that provides evidence (sometimes stories, sometimes numbers) to support the design that most strongly supports the agreed goals.

Goals. That’s one pre-requisite you need for this experimental approach to work. You need to have agreed what your goals are for the design. What success looks like. Without this agreement, no change to methodology will save you.

The experimental mindset is an egalitarian approach to design. It allows that anyone can suggest a design solutions and, rather than argue endlessly about whether it is better or worse than other approaches, you design a test. Find out how to find out which design works best.

Hypothesis, prototype, test.

There are loads of tools you can use to test ideas quickly – from some quick in person user research, to some A/B testing (if you’re not set up to do A/B testing, meet your friend Google Content Experiments and get onto this immediately), to an online card sort, to one of the range of tests that places like VerifyApp offer. The methods for testing are limited only by your creativity and are mostly inexpensive.

Sure, you can’t design from the ground up this way – you will still need a good designer that you trust get you to a good starting point from which you can experiment up, but once you’ve got the framework in place, don’t waste time and goodwill bickering about the details but encourage experimentation throughout the entire organisation. You’ll raise the overall ‘design IQ’ and happiness quotient of your company, your design team and, most probably, even yourself.

Design is the easy part…

On approach, I’m warned by most clients that this will be a very tricky design problem, very hard to get right and of course, utterly imperative to the business that we do so.

And, at first glance, often this appears to be the case.

It’s been my experience that the main reason most designs go unsolved is not the lack of talented designers or their interest in solving the problem. Instead, the problem is with the organisation themselves  – their inability to allow themselves to implement the right design, or even any good design.

Many times I’ve suggested a design approach only for the in house designer on the team to literally pull the design from their desk drawer or computer and to tell me how they tried to get the organisation to go this way two, three, maybe four or five years ago. They tried and tried, had no success, and filed the design away so they can get on with the work the organisation deemed acceptable or appropriate. It’s kind of depressing, and almost embarrassing when my main role is to advocate for work that was actually done years before I appeared. And sometimes it works.

Politics and egos are the main reasons that great design goes awry – either it is never presented (because presenting it is a risk to those egos and would be not wise politically), or it is presented and dismissed, or it is presented and then changed such that egos are not wounded and the politics are in tact, the design integrity is hardly a passing consideration.

Organisation processes and complexity are another common killer. As more and more, the digital products replace the previous products and functions of the organisation, this requires a transition in how things should be done that most organisations are unprepared for an unwilling to support. They’d rather keep doing things the way they always have, and craft a design that doesn’t trouble their processes or require additional resources. You know you’re designing for an organisation on the way out the back door when you come across this – disrupt yourselves or be disrupted, Peter Drucker, amongst others, has been telling us this for half a century (or more). Still, it can be surprisingly hard to do. We don’t like change and the changes required often threaten the existing egos and power structures. See above.

At first glance, the solution is strategy. Get more designers higher up the food chain and involved in the creation of strategies that would guide an organisation to make better decisions. Sounds right, but the reality is different. Most places I encounter these problems have all kinds of strategies talking about how important design and the end user is to them. They all handwave the right way, but the execution doesn’t match the strategy. This is the reality we live in – almost every organisation you come across is loudly proclaiming their interest in the customer experience and surveying you within an inch of your life to prove it. They’re talking about the importance of design and hiring expensive designers (who are then nobbled by the organisation). None of this matters if the execution, the tactics, don’t fit the strategy. And most often, it doesn’t.

I’ve tried approaching this two ways – firstly playing the politics and trying to get involved higher up, spending lots of time in meetings, or secondly: just executing – making things that actually live out the strategy that mostly lives on posters and induction manuals and giving the higher ups a better choice to make, giving them a good choice to make not expecting them to get there on their own and then brief the design team. These days I don’t get too much feedback throughout the design process (forget wireframes) – make it and then iterate. It’s been the second approach that has worked better.

‘Show, don’t tell’ is a design principle that seems to work well in design practice as well.

It saddens me how many great design solutions are hidden away in filing cabinets. It’s not enough to know the right answers, the real design challenge is in getting the organisation to adopt and implement and maintain (a whole other challenge) good design. It feels to me like we  need to focus on this more.

If the government can do it….
(a love letter to Gov.uk)

Earlier this month the UK Government Digital Service publicly launched the gov.uk , the ‘single government domain’ or the primary interface for UK Government’s digital interaction with citizens, replacing sites including DirectGov and BusinessLink.

Although I’m no expert on public sector projects or the history of the UK Government’s web presence (I’ve done bits and pieces as I suspect many of the UX Community in UK have done), I want to take a moment to commemorate the impact of this achievement for anyone who is trying to encourage large organisations to embrace better digital work practices.

This is a big deal.

It’s important because Gov.UK arguably brings a new high standard of design, content and overall user centricity to public sector digital projects. It’s true that the UK Government has engaged its share of designers and user experience (or, probably more accurately, usability) people over the years, until now it has felt as though they were constrained to making things less bad, rather than aspiring to really create experiences that citizens wanted to engage with.

That’s because this is not really a design case study – it’s not about the government finally finding a decent designer to pretty up the interface or a usability person to write the perfect report telling them what to do. It’s about actually creating an environment where, having hired those people, they are able to do what they are good at and to actually get their work, relatively unscathed, through the complex web of stakeholder engagement and approval processes, and into ours – the citizen’s (or in my case, resident) hands.

What the Government Digital Service have given us is a brilliant case study in overhauling the way things were done before and changing them around so that they can support the creation of better user experiences online.

I thank the @GDSTeam for giving me the case study I need to present to large complex organisations who are trying to revolutionise their  user experience without changing the way that their organisations work. Now I can say,  ‘Well, if the UK Government can do it, I’m sure we can’. In my experience, it’s quite  compelling.

A page like this doesn’t come into existence because one designer had a good idea. This is no vanity redesign project, these designs and this content has gone through the complex series of stakeholders and approval processes to get from ‘good idea’ to ‘actually live’.

Being able to sell something as radically different, to give stakeholders the confidence to go with something like this -that is a tremendous achievement.

Remember – this is the typical approach to public sector content:


This is not a story about interface design (although kudos to the designers who have worked so long and hard on this project). It’s a story about organisational design. The changes that the GDS Team made to how digital design is done in government is what enabled design like this to emerge.

Changes like:

  • moving to a centralised, multidisciplinary team who work in close proximity and are able to focus on solving particular problems, not get hauled around from project to project to project with no time to focus.
  • housing this team in a space that facilitates close teamwork between the members of these small, agile teams (including, from what I’ve seen, plenty of wall space. It matters!)
  • using an iterative but agile project methodology that involves regular testing information gathering allowing the team to make decisions driven by data rather than opinions
  • working openly, sharing what they are doing (including the code) and why they are doing, inviting others to participate in the process and inviting feedback often.
  • having clear and inspiring leadership who continue to evangelise for the team higher up in the organisation and be the battering rams driving change throughout the organisation.
  • having vocal and consistent support from the highest parts of the organisation
  • spending time on creating artefacts that allow the team, as it grows, to maintain a clear shared vision about the way they are approaching challenges and defining solutions.

and many more I’m sure.

More than anything I’m thankful for the final point – the openness and the time spent creating and sharing artefacts.

From the very beginning, the team have been sharing their methodology and rationale, their project documentation and even their code. They have been helping to enable the rest of the world – not just governments – to improve their practice and make better digital products.

Some of the treasures that they’ve provided us with include:

There is plenty to criticise, there always is. Nothing is perfect, and even less so in large and complex projects like this. And yes, the real challenges are ahead – can this scale and can it be maintained for the years to come now that the ‘launch’ has passed.

Most of all though, here is an amazing opportunity for all of us – public sector or otherwise, UK and around the world, to take advantage of the awesome work the team has done and the resources they’ve provided us with and to use them ourselves to no longer accept ‘the way things are done around here’ but to require and facilitate transformation.

The space you work in, the size of your team, the access to and interest from upper management, your project methodology – all of these things and many more will directly impact your ability to do good work, to deliver good experience. If you want to fix the experience, it’s critical to look at the environment that is impacting the ability of your team to deliver.

People often talk about Apple’s design process, but I think equally important is the way that Steve Jobs took the focus off the Profit &Loss statement- making that the responsibility of just one person and, apparently, running just one P&L for the world’s most valuable company. (Most companies run multiple P&Ls between departments (functional or product), and crazy decision making and politicking ensues).

Only through transforming the way your team, your organisation works will you really be able to transform the experiences that the organisation is creating for its audience. It’s not a UI problem, it’s an organisational design problem. Those things do matter.

So, get stuck into addressing the environment as well as the experience design and when you’re feeling challenged, remind yourself and your colleagues, ‘well, if the UK Government can do it…

Client/Agency Engagement is F*cked, Waterfall UX Design is a Symptom

Ross Popoff-Walker wrote a properly ranty blog post yesterday entitled ‘UX Design at Digital Agencies is F*cked‘ in which he discussed the typical waterfall methodology utilised by digital agencies he’s worked in.

Most of us with any agency experience would have no doubt been nodding in agreement to read:

Big digital agencies especially, will kick off a project with a “discovery phase” (which may or may not actually discover anything), and quickly jump into a waterfall-style design process of UX sketching, wireframing, and client meetings/approvals. Then after many (many) rounds of visual design… and only then… will agencies start to move into the development and tech stage. Only after every pixel has been pushed and use-case documented, will something be made that is working and actually functional.

Developers and tech leaders intuitively get the problem with this. Websites (or anything digital) are not buildings, made the stand the test of time without change — they are meant to be tested and iterated, and improved continuously. But in my experience, it has never made anything of real value to a client.

Ross goes on to advocate that agencies take up the Lean Startup methodology widely in use amongst start ups and some of the more forward thinking and/or buzzword aware larger companies. I concur. This is indeed a fine and very user focussed way to approach a project.

However, Ross glosses over the reason agencies work this way (‘comfort, dogma, and the ease of billing clients’ he suggests). I think a lot of agencies want to work in a more Lean or Agile way (and some attempt to do so), but the nature of the agency/client engagement will have to change substantially in order for this way of working to become widely adopted.

A few things happen when you hire an agency.

Firstly, the client effectively outsources the work.
They create a separation between themselves and the people who are doing the work.

Even the agencies who work most closely with their clients (and by this I mean properly in each others faces physically or virtually ALL the time). This creates a different dynamic than what you get in an inhouse team. There is an “us” and a “them” and they have very different realms of expertise and knowledge and often not a great way of combining these two sets of knowledge to make a great product.

The lack of integration between the company who needs the project done and the company who is doing the project creates a very different shape to a typical (effective) Agile or Lean team, and it makes it difficult to work effectively.

It also introduces another ‘customer’ to the mix – one that is not the end user customer, but one who will sign off the project and pay the bills – so, probably, a more important customer to the agency than the *real* customer that the project is being created to serve.

Complicated huh. Makes it hard to focus on what’s really important when there are actually TWO things, often in conflict, that are important. Agencies will always preference making their customer happy over making the customer’s customer happy. That’s understandably, but it doesn’t lead to good project outcomes.

Secondly, when the client outsources the work, they feel as though they’re outsourcing the risk.

They effectively pay a premium for an agency who knows what they’re doing to do that thing well. It tends not to play well for an agency to then spend the duration of the contract being actively uncertain, making hypotheses and validating them, using the client’s money to ‘learn’.

This, traditionally, is not what we pay a top class agency to do. We pay them to know stuff and to get stuff right, and to be the people we blame if it doesn’t work out well. Until clients get comfortable with this (will they ever?) it will be difficult, nigh impossible, for an agency to be properly Lean or even agile.

Thirdly, when you’re paying an agency a lot of money (and you usually do), you want to feel confident about what you’re going to get when then money is spent.

This is why clients are so desirous of spec work in the pitch process – it makes them feel more confident about what they’re going to get for their money. Getting them to let go to spec work in the pitch is hard enough, how much luck do you think the Biz Dev guys are going to have selling Lean, where all we have is a Build, Measure, Learn process that admits we don’t really know anything for sure, and the possibility of pivots along the way. (Not to mention that most biz dev guys probably don’t have the first idea what Lean is and have the wrong idea about Agile).

No one ever got fired hiring a big name agency to do waterfall, complete with functional specs and three different visual design variants for the marketing team to choose from. They probably didn’t get a good product at the end of the process either, but they got a thing that looks as though it probably took as much time as the agency said it was going to take, and looked kind of pretty, and so they don’t feel ripped off and angry. And they won’t get fired.

It takes a special kind of client to take the risk and develop the level of trust and integration required to work the way that Mr Popoff-Walker and many, many other inhabitants of agency world would like to work.

The agency model is certainly pretty broken, but both agencies and – I’d say more importantly – clients need to take responsibility for that, and take both action and a little risk to help mend it.