I’m sure that many of you will have heard about the very worthy Alpha.gov.uk project, the first prototype of which was released earlier this month.
If you’re a user experience practitioner, this should particularly interesting to you.
By way of a quick background, the AlphaGov project was formed in response to findings from a report by Martha Lane Fox, Revolution not Evolution into Government online services and opportunities to improve. (As a tangent, I’d love to see her in a cagefight with Lou ‘The redesign must die‘ Rosenfeld)
In this report she recommended the introduction of
“a service culture, putting the needs of citizens ahead of those of departmentsâ€
The AlphaGov project responded, setting out two overarching objectives:
- To test, in public, a prototype of a new, single UK Government website.
- To design & build a UK Government website using open, agile, multi-disciplinary product development techniques and technologies, shaped by an obsession with meeting user needs.
See. It doesn’t get more UX-interesting than that right? It reminds me quite a bit of the D7UX project I worked on with Mark Boulton and the Drupal community, so I’ve been following it’s progress with a keen interest.
Now, go have a play with the prototype and see what you think. I’m actually not going to comment on the UX of the prototype today, partly because it’s actually quite difficult to do so. I’ll get to that later.
What I want to talk about today is the responsibility that playing out a project like this in public brings with it and how, in my opinion, AlphaGov have let down both the UX and Inclusive Design/Accessibility professional communities.
What you do, not what you say
Let me start by saying that I have a lot of admiration for the the ambition of this project. There is a lot that is good about it. There are also a lot of smart and talented people on the team.
The first thing that strikes me as strange is that on a project that claims to have an obsession with meeting user needs, the team contains a visual designer and a content strategist, a general strategist and multiple search analysists but NOT a user experience lead.
That’s right. We have an obsession with meeting user needs but not so much that we’ll actually hire someone who has extensive experience in actually making that happen.
Now, the project was fortunate in that it had Richard Pope, who I first met when he was a very UX-savvy developer at Moo and who played the Product Lead role on AlphaGov. As far as UX resources go, you could do a lot worse.
The team also recruited Paul Annett later into the project. Paul also has some UX experience but, as I understand it, his role was primarily as visual designer, making the interface a nicer place to be.
Without commenting on the interface itself, the lack of a rigorous approach to user experience is very evident in the way that the team talk about the work that they have done and their ‘design rules‘.
In a recent blog post about their agile methodology Project Manager Jamie Arnold describes exactly what this ‘obsession with user need’ entailed:
We spent the first two weeks in February recruiting a team from inside and outside of government, talking through the scope, agreeing some design rules and agreeing a vision for the Alphagov product based around the recommendations of Martha’s report.
We ended those two weeks with a list of prioritised user needs (based around search analytics from Directgov, Hitwise and departments),
We roughly grouped into functional areas and stuck to the wall. Each card (or user story) represented a user need, prioritised roughly from left to right and top to bottom.
Crucially also there was a fair amount of @tomskitomski and @memespring‘s product experience. All this was more than good enough to get on with twelve weekly development sprints.
More than good enough, eh? For many projects this would have been more than they ever had to work with.
But this is not just any project. This is a groundbreaking, whole of government initiative that claims to take a User Centred approach and be obsessed with knowing and supporting the end user need.
I think a project like that needs to demonstrate User Centred-ness a little more rigorously. For example.
Who is the audience?
At no point that I saw did the AlphaGov team ever apparently think deeply about what kind of an end user they were going to prioritise. They talk about ‘thinking about who our users were’ and having a ‘user-base of all the entire adult population of a country’.
As User Experience practitioners we know that although you might want the whole country to use whatever you’re designing, you need to put a ring around the kind of users you MOST want to support.
As designers we always privilege some behavioural attributes over others, even if we don’t articulate it. By not thoughtfully articulating this, you risk either an incoherent approach to the experience design or resort to self-referential design (designing for your own behaviour – something that is incredibly difficult to overcome).
You can’t take a User Centred approach to design when your user is ‘Everyone’. You need to define who your users are. You must clearly identify the behavioural characteristics that you most want to support and focus on designing to best support these.
There are many valid design approaches that do not require such a clearly articulated definition of the end user, but these are NOT user centred approaches. Thinking generally about ‘users’ while we design is not doing user centred design. I think it’s pretty irresponsible to suggest that it is.
AlphaGov sends a message that you can say you’re doing User Centred Design but you don’t have to show any evidence of a UCD process – audience definition, research, user involvement, design principles that actually track to specific behaviour attributes.
For example, it would have been great to see some personas developed and shared for this project – even hypothetical ones that drew on the data/insight available to the team. As well as helping the team avoid the problem of the ‘elastic user’ (particularly problematic when you do think your target audience is everyone), it would also help us be better able to evaluate what is good and bad about the prototype. It would also have demonstrated that the team was actually practicing User Centred Design.
(Elastic user, for those not familiar with the term, was coined by Alan Cooper to describe the way that while making product decisions different stakeholders may define the ‘user’ according to their convenience, often resulting in self-referential rather than user-centred design. More here).
This leads us to one of the complexities of the AlphaGov audience which is that, in reality, rather then being obsessively user-centred, the project had two competing audiences. The largely undefined end user and, often more importantly, the stakeholders who would ultimately decide the fate of the project – public servants. These two audiences have VERY different motivations and goals for this project, and the impact of the latter on design decision making was never clearer than when the accessibility topic came up.
On Accessibility and a conflict of interest
Again, from what I know, there was no formal expert accessibility (or inclusive design as I prefer to call it) consultancy on the project team. This is not to say that the team didn’t have quite a bit of knowledge about the mechanics of accessibility (the impact of technical decisions on whether something could be certified ‘accessible’).
The team sets out a really thoughtful understanding of what it takes to make a service properly accessible:
Accessibility should start with research and consideration, not with box-ticking or sprinkling a few standard accessibility features – especially in a government context where a user journey regularly extends into the real world (Booking a driving test? You’ll probably want to know the facilities at the test-centre).
Ultimately, the AlphaGov prototype doesn’t make any significant attempt at achieving accessibility (particularly making a site that works fine even with JavaScript is switched off) apparently due to the short timeframes and ability to ‘retrofit’ accessibility later (hrm).
Actually, what I picked up from discussions about this on Twitter and elsewhere was that actually, it was the other target audience – the stakeholders – who most influenced this decision. If they put the focus on accessibility, they’d have to take away some of the ‘shiny’ – AlphaGov would end up feeling like Just Another Government Website. Rightly or wrongly, the shiny would help excite the public servants to approve and fund a beta version of the prototype.
Perhaps it was a noble sacrifice… who knows. Point is, it’s far from transparent.
The message that the world takes away from this exchange is that accessibility, even when your audience ‘entire adult population of a country’ is optional. And that accessibility can be ‘done later’ not, as they had first set out, built into design considerations from the outset.
These are really bad messages to be sending and, given how publicly visible and lauded this project is, sets the work of many amazing inclusive design specialists back considerably.
It’s hard enough to sell in good accessibility work already. AlphaGov just made it harder.
Activity Based Design and Search Analytics
OK. So I will talk briefly about the prototype… but mostly to discuss how the data you have access to can significantly shape your design.
The team have published very little information on the data that has guided their design decision making for this project but we do know that search activity has influenced it heavily. A team of sixteen people included no UX lead (sorry, did I mention that already?) but two people doing search analysis.
In the design rationale blog post, Richard Pope implies that search logs strongly influenced the design and information architecture strategy for the prototype.
we spent the first couple of weeks scouring search logs and analytics for the various central government websites; thinking about who our users were and generally discussing the kind of thing we were setting out to make
Based on what we learned from looking at search-logs, we knew that there was a relatively small subset of tasks that require the majority of people need to interact with government online. So we should do less and focus on tasks.
Since for the vast majority of people their web journeys (finding out the date of the next bank-holiday, or reporting a lost passport) start with a search engine rather than a direct visit we should think of Google as the homepage and we should also feed Google, Bing and other search engines nice friendly urls.
If someone is just landing at a page on your site then it’s helpful to start thinking of every visit being a new user, assuming they have no prior knowledge of the structure or content website they have landed at.
It is really difficult to evaluate this prototype from a user experience perspective, given the competing target audiences. The best you can do is try to recall the last few times you interacted with a government website and try to reenact that here. Every time I do that I come away with a lingering feeling of disengagement. There’s something that search logs probably don’t really show which is that this is MY government. For better or worse, I have a long term and multifaceted relationship with this government and yet, every time I encounter this site it (by design) makes me feel as though this is my first visit. I find that really unsatisfying and kind of perturbing.
Now, this is not a professional design critique, this is a qualitative research data point of one. But it’s not something that you’ll ever pick up from search stats and analytics. I could bore you further with how I find the promise of localisation with this infinite noob status even more perplexing, but you’d have to spend time with me to understand it. And then spend some time with a bunch of other people to see if this is a common theme or just me being an edgecase.
And people will never post this kind of feedback on GetSatisfaction. (Most people can’t really articulate this kind of weird feeling and wouldn’t think that it was important enough to comment on compared to, say, a bug. You need a good facilitator to extract this kind of data).
To do really good user experience design you need a mix of data points. If you privilege one set of data, you’ll see that in your design. I think we’ve got some of that going on with AlphaGov.
Quantitative data is fantastic. I’d love to see more of what the team had to work with and how they applied it in their design process. But it’s just one kind of input. Qualitative research helps you better understand your end users and thereby to design better for them.
People who do User Centred Design do Qualitative Research.
User Experience is a Time Soak/Non-Agile
Which leads me to the final problematic sub-text that I felt emanating from the AlphaGov team which was essentially that ‘we’re as user centred/accessible as we can be given that we only have 10 weeks to build this thing’. This perpetuates the myth that User Experience can be a time soak, that it slows you down, that it doesn’t really have a place in an Agile methodology.
This is where having an experience UX practitioner on the team from early on could have been helpful.
It is certainly true that historically, Agile and UX have had a fairly vexed relationship but these days many practitioners are experienced and adept at including both user research and ux design into the most demanding agile iterations. We have a toolkit of lightweight qualitative research approaches that work beautifully in this kind of fast paced and responsive environment. UX does not have to be a laggard either at the outset or in the throes of an agile project.
The ten week project timeframe is absolutely no reason to not include real practice of user experience in the process. You may need to find someone who has experience working this way (not all UXers find this kind of project much fun), and you may need to be creative in the way you structure the work, but you should definitely be doing it. Particularly if you’re setting an example of how projects should be done, as the AlphaGov team certainly was.
In conclusion
I want to repeat again, this is a very worthy project and many of their design principles are, I think, sound. For many commercial projects, the methodology that they’ve applied and shared is absolutely appropriate. But the bar is set higher here.
By doing this project in public, by making such a big deal of putting the end user needs and their importance to the project, the AlphaGov team have set themselves up as rolemodels. They’re sending messages about the the way things should be done. They’ve made quite a rod for their back.
If I was just a member of the community, I’d probably be nothing short of delighted with what they’ve achieved. Unfortunately, as a User Experience practitioner, I feel kind of glum. This project has talked the talk of caring about the end user, of placing their needs at the centre of the project and above the needs and desires of government, but at every step, they’ve done little to set a good example for how others should actually do this.
I hope AlphaGov does move forward into BetaGov.
But I hope, if they do, they take a moment to think about what the public performance of AlphaGov and then BetaGov means for their professional community.
Either stop calling the project User Centred, or hire someone to really focus on user experience and do more to share how they’ve integrated real user insight into their design strategy and implementation.
There’s a big opportunity to set a good example to a big audience here. Let’s take advantage of that opportunity and show the UK Government, digital industry, hell, the whole world what projects really look like when they’re user centred, – that they don’t have to be cumbersome, expensive and slow.
Imagine that, a properly user centred government website that was agile, and shiny and amazing. Now, that’s something to get excited about.
I used to do loads of usability testing for government websites (and I’m a qualitative researcher by background), and what jumped out at me there is that search strategy can be really misleading – people will game their search to locate things that they know must be in there but which they can’t actually find.
It bothers me that users aren’t represented in there, because web-savvy government social media experts are typically about as far from the average user as it is possible to be.
[To clarify I was part of the team in a product lead / UX capacity – strictly no coding allowed (although i did sneak in a bit when no one was looking). Plus several members of the team have spent years doing government related stuff (mysociety etc), so we weren’t coming to this totally from scratch.]
Great blog post. I agree that you could argue, in the strictest sense of the term, we were not user centered in the design-process, but I would dispute the idea that we were not being user centered in the product we were aiming to build.
Yes, we spent a long time looking over search logs form various government websites and also tapping into the brains of good people from within government identifying what people want from government online.
In terms of defining users, the fact is that the user base is everybody and there’s no escaping that, this is the government. The approach we took was, rather than to identify user types, to identify the main things people were after from government. To identify the tasks (each of which has many different modes and user types) and build the best possible solution for that task and user.
If you take loosing a passport as one example – that is a specific task (the law dictates what you have to do), but the process is different if you are at home or abroad. So we built two variants on the tool and user GeoIP to work out which type of user you are.
On the accessibility point – there are a couple of elements of the site that do not work without javascript (a decision that paid off in terms of being able to iterate design quickly), but when compared to many other government and commercial sites I think we come off quite well. There is also quite a lot of good practice in there.
The reason for not taking the ‘ticking boxes’ approach is we did not want to claim we had solved accessibility in the government context, rather we wanted to make it clear that it is the beginning of a process.
This may not have been the right call, but it feels right to open up the debate about what accessibility means in the government context rather than just following the standards and leaving it at that.
If we had done one thing extra in the approach to both accessibility and UX what should they have been?
I’ll try and get some more of the thinking and practice behind this blogged next week.
Thanks again!
Well done Leisa. Great post that neatly articulates the challenges and the shortcomings of this project.
Let’s hope that it’s early enough for the projec to find its way to being truly user-centred and universal.
I’m sad to see that Richard Pope’s reply supports your arguments. “the fact is that the user base is everybody and there’s no escaping that, this is the government”. No, really it isn’t. The user base for different bits of government varies a lot, and that has to be thought about.
For example, in developing the Self-Assessment tax return, we were in theory catering for everyone, but in practice there are easily and clearly defined different user groups with different needs. You may think “but I don’t like the tax return” and I’d agree with you, it’s got lots of issues. But that’s a whole other story – mostly, because of all the stakeholder problems that you identify.
Best
Caroline
Interesting post, and comments!
I agree that (as a UXer) the talk of user-centred design was just that – talk. I’m not sure that the decisions during the project were necessarily wrong, but labelling the process as UCD wasn’t right.
To hop over the fence though, in a 10 week project I’m not sure how you could usefully define all the audiences for this scale of website?
What I’d have liked to have heard more of was how the building blocks were being put in place so that it could be accessible and user-centred in future.
Caroline’s example is a good one: You could tackle a tax-form as a ‘block’, research the audience and refine the task(s) and then refine that part of the site. Then tackle the next block…
From an accessibility point of view, if they were putting in place APIs which had sufficient content & attributes to make a (not yet existing) website accessible, that is the important aspect at this stage.
For example, Apple put in the building blocks of accessibility into iOS pretty much from day 1, even though there was no applications (e.g. VoiceOver) to take advantage of it then. Now it’s (relatively) easy for developers to make accessible applications on iOS.
I don’t think there’s much point in arguing about it working without JavaScript (see http://webaim.org/blog/javascript-as-an-accessibility-concern/#comment-63498), but they would be in a stronger position if they could show how it had been thought about upfront.
One thing I am very positive about – it is something of a wake-up call for Government departments that are stuck with decrepit and expensive digital products. The use of modern frameworks and development methods is long overdue in Government.
Totally agree that this project is A Good Thing to see in Government and I really hope that it moves forward to Beta and beyond.
I don’t think we needed to define every single audience for the website, I agree, that would be nuts to attempt for a 10 week project and possibly nuts (and unnecessary?) to attempt at all!
What I would like to have seen was some kind of audience positioning so that we can better understand the kind of people this website is designed to best support (understanding, of course, that ultimately everyone is supposed to be able to use it.)
For example – it could be designed to suit the people who most frequently interact with Government services online and aim to make these interactions as efficient as possible. These people might have a fairly high reading age/education level and possibly undertake tasks for themselves and also for other family members and business.
Or, perhaps, people who most need to access Government services both widely and frequently actually have the lowest English reading ages and technical literacy…
Even these three criteria: reading age, ‘government’ literacy (how well people understand the way the Government is structured and the services it provides) and technical literacy – to make a mark on the continuum from low to high on each of these three criteria and say ‘that’s who we are MOST supporting’ would be an excellent first step, right?
I’m always mystified when people define UX as something separate and distinct from other roles on a team. For my money, Chris is right.
And anyway, surely this stage of the project is about thinking through making.
What would you have done differently had you been involved?
Matt, I’m not sure when I defined it as something separate and distinct from other roles, that certainly wasn’t my intention. I do think, however, that having someone on the team who has a strong background in UX would have been valuable. If this person can perform multiple roles on the team, all the better.
In particular, someone who has experience in helping to define and frame target users and to conduct, aggregate and act on qualitative and observational research. In addition to help making prototype, this person would be able to create some tools to help frame design and decision making allowing the team to make better decisions more quickly.
That’s what I think I would have done differently if I’d have been involved.
This is not really the point I was trying to make in this post tho’. I don’t want to suggest that all projects need a UX person or even that skillset at all times but if you call your project user centred and say that you’re obsessed with meeting end user needs, then I’m going to expect that you’re doing something like what an User Experience person might do.
I actually used the site and find it extremely useful, friendly and full of utility. I think any further “UX engineering” will actually spoilt it.
Hi Leisa,
Thanks so much for writing this blog. I have been leaving comments on the AlphaGov team blog about accessibility and Tweeting about it too for the last three days. I have also emailed various contacts ‘in the know’ about it too.
You really have covered all of the issues here though and without repeating what you have said already I would just like to add a couple of points in agreement here!
Firstly, I find that the blogs about accessibility and Agile on the AlphaGov site have a slight air of arrogance about them and I think that’s what is really getting me the most about this project. Like the rest of us have no idea how hard it is to run a design project.
Perhaps the team have got a bit carried away with themselves because they are working on such a high profile project for the UK. However, if this site is a UX failure those bubbles may be burst very soon folks. Perhaps by a legal case – see Canadian Gov for reference.
Secondly, I worked as the accessibility specialist for the BBC for several years – our audience was indeed the UK or to refine it slightly, anyone with a TV license. However, Caroline Jarrett is totally correct when she says that even when it seems you are catering for ‘all’ “in practice there are easily and clearly defined different user groups with different needs.”
This is something that me and my user experience colleagues at the Beeb consistently repeated to product teams and web producers.
There are some areas of the site that will need to be more accessible to certain groups of people e.g those users viewing information about disability benefits are more likely to have a disability of some form so they may required accessibility features of some sort.
Alternatively, it is unlikely that you will have a large number of blind users applying for a driving licence online. However, they may be interested in the Motability scheme, Freedom pass information or access to work funding (purely an example).
Thirdly, Richard Pope may be a ‘savvy developer’ but he hasn’t shown me any true understanding in his comment on this blog or in answer to my own comments on the AlphaGov blog – sorry Richard, it’s nothing personal and I am very aware that this project isn’t your full responsibility.
However, the JavaScript argument is not working for me so drop it. JavaScript, alt tags and good HTML are small parts of accessibility – JavaSvript causes few problems these days. It certainly doesn’t slow down accessible design iteration.
Fourthly, I have worked in many Agile project teams and 10 -12 weeks in more than enough time to recruit three sets of disabled users and get them in to have a go with the site or involve them in design workshops.
Finally, and I suppose this is directed at Jamie Arnold; standing around a board making up user stories and asking stakeholders or ‘good people in the governement’ (Richard) with ‘a fair amount of product experience’ is not good enough by a long chalk. It’s simply lazy!
What they conjure up in their heads on that day has no relation to what an actual user may want from the service yo are building. How could they ever represent the 20 years old female uni graduate with cerebral palsy, the 35 years old professional male with arthritis, the 29 year old single mum with learning difficulties, the 80 year old woman with dementia, the 16 year old school leaver with dyslexia, the 50 year old male job hunter with Aspergers or the 45 years old, recently redundant IT consultant who is blind?
Using metrics and search logs is not going to tell you what a real user can tell you. They will simply tell you what searchers searched for and what relationships the software found in those searches.
Leisa you are so right when you say that this has made it so much harder for us accessibility lot. It has taken us back about 10 years and I am gutted because I had high hopes for this project.
>> Using metrics and search logs is
>> not going to tell you what a real
>> user can tell you.
Damn right! What if a user cannot perform a function due to a bad site design, will that turn up in the logs? The logs will tell you how people are currently using your application and not how they want to be using it. It is a good source of information but definitely not enough in isolation.
I am coming at this from, a “what data is available” angle. I think the comments on logfiles being an incomplete picture are very apposite. A log file can tell you all ther things a user has tried but not what they were trying to do when they tried them.
Even something like the 4Q questions to see if people had succeeded in their missions would have provided the start of a qulatative data feed. getting real people in and using them to find out what they tried to do and how they tried to do it would be another feed of useful data.
This looks like a classic case of gthe Macnamara fallacy using the information they had to hand and deciding it defined the universe. It is the job of a UX expert to step beyond “what data do we have” and to look at “what data do we need”. Perhaps that is the single most compelling argument for giving someone on the team that role?
Rufus
P.S. Lucy the BBC remit covers more than just people with licences. If you listen to Radio 4 but do not watch TV you are part of the group that needs representation.
Paul Annett was on the AlphaGov team and claims to be a UX designer.
Does Google identify users in the way you seem to be suggesting? From the few presentations they’ve done, it seems they focus more on tasks. Isn’t that what the AlphaGov team did?
Very interesting post Leisa.
Given that DirectGov employed UX specialists in the past, I would have thought there was quite a bit of qualitative data kicking around already. Maybe there were some pre-existing personas that could have been re-purposed as a baseline. If so, perhaps they can still be fleshed out for the beta phase, by adding data from user testing?
Alphagov was a very bold project and approach for UK government, not least the adoption of Scrum. I also really welcomed the transparency. Inevitably any project taking such a public approach as Alphagov in the current political climate is going to have its successes and flaws laid bare for all to see. Which is a good thing. It’s perhaps a shame there wasn’t the opportunity for more challenge around the UX approach at the very early stages.
I have to admit I had assumed that the Alphagov team were recruiting in users to do some guerilla testing throughout the project. Personally I would really like to see the project validate some design patterns that can be re-purposed with some clarity on why/when they work and for what type of users. I do hope there is still time and budget to start doing some evaluation of the prototype.
Brilliant post. If it goes ahead at all, Alphagov will fail in the same way Directgov failed – it doesn’t tailor design and content for specific audiences. A service on a government supersite will always be less useable than the same service on its own website, designed for that purpose. This is a vanity project which has already run to hundreds of thousands of pounds. Why we’re taking strategic cues from someone who makes a living from online Karaoke is beyond me.
Richard Pope’s comment concerns me:
From my personal experience with user testing almost 50% of participants simply don’t use site search. Now this might not be reflected in a large scale study, but many people I have asked state that they never use site search. This being the case, his ‘relatively small subset of tasks’ might be quite wide of the mark.