D7UX User Research & Usability Testing – Proposed Approach

The best way to a good end user experience is a program of iterative testing and design. There’s been quite a lot of design but not as much iteration as we’d have liked, so now is a good time to really kick off some testing of the D7 interface as it comes into existence so that we can potentially iterate it a little before it is set loose on the unsuspecting public.

In order to take best advantage of the enthusiasm within the community to participate in testing we should ensure that we are taking a generally consistent approach to the test process so that we can better compare our results and make decisions more easily. A way that anyone who wants to can conduct some tests and provide feedback, insight and recommendations, no matter where they are in the world, in a way that we can actually act upon.

Here are some initial thoughts on a framework for testing the D7 interface that can be adopted by anyone within or outside of the community and around the world.

Making use of previous usability test framework:

The Drupal Usability group conducted formal usability testing in Baltimore in 2009 (ref: http://drupal.org/usability-test-university-baltimore-community-solutions) which has informed much of the UX work that has been undertaken over recent months. It makes sense to capitalise on this previous effort as far as possible, not only to reduce workload at this point but also because it will provide a useful benchmark. There are some areas where it is appropriate to make some amendments to the approach taken in the Baltimore testing, these will be discussed below.

Target Audience/Recruitment:

As with the Baltimore approach, our testing should target two specific user groups who are important to achieving increased ‘reach’ for Drupal, and who have been at the heart of the D7UX design effort. Baltimore identified these groups as ‘Beginner’ and ‘Intermediate’. This assumes that every end user is at some point on the same Drupal learning curve, which our research has shown not to be the case – this is demonstrated in the example users identified in ‘Who Is D7UX for’ (http://www.d7ux.org/who-is-d7ux-for/), where the users Verity and Jeremy are described.

Verity would equate most closely to the ‘Beginner’ audience used in the Baltimore testing, however the description of ‘Beginners’ it was specified that these people have some development experience, and the script implies that their role may involved building a website using Drupal. Verity, on the other hand, is the recipient end user of a Drupal site who is then tasks with maintaining the content on that website. This is a very typical end user role for Drupal and one which we hope that Drupal 7 will do a much better job of supporting.

Jeremy, however is quite a good fit for the Intermediate audience group recruited in Baltimore, although again, development skills should not be required in order for him to successfully complete the tasks in his test script.

I propose that went we are recruiting people to participate in our user research we compare them to the Verity & Jeremy sketches and if they are similar, they are good matches for testing. Depending on which they match, we will then use a discussion guide and set of tasks tailored to their role and objectives.

Discussion Guide/Script:

The scenario presented in the Baltimore testing was essentially to build and/or manage content on ‘a website for a conference where users will post information about the conference like the schedule, speaker bios and local bars’. This is a great scenario and is suitable for both our Verity and our Jeremy participants and I suggest that we re-use this scenario, the content materials prepared to support the scenario and used in the test, and as far as possible, the script that was devised for the previous round of testing.

The existing beginner script is, I think, almost entirely suitable for the Verity role, and comprises the following tasks:

  • publish some information about the conference (to existing content types/sections of the site) taking content from a text file provided.
  • create a navigation link (if we leave anything out of this script, I think this might be it… not sure if this is a real Verity task… could be tho’)
  • change the look of your site, remove the logo at the top left
  • creating a new role (speaker) with additional permissions (suggest that perhaps rather than creating a new role, assigning users to a the speaker role might be more appropriate for Verity)
  • create a new page (schedule) and navigation to that page

The existing Intermediate script is also very suitable for Jeremy, and comprises the following tasks:

  • installing drupal
  • publishing some content about the conference to the site
  • creating navigation
  • creating a new role (speaker) with additional permissions
  • using taxonomy to categories ‘workshops’
  • using a block to highlight content on every page of the site
  • customising the URL of a particular page
  • allowing speaker role to use images in their content creation
  • set up site search

There may be some additional tasks that we want to include, however these scripts are already quite lengthy so perhaps it is best to prioritise tasks and we can iterate the script over time as more testing is completed, to increase task coverage.

Test Materials (What to test on)

I think this is the biggest challenge, especially in the case of Verity, and it’s not something for which I have an immediate solution.

Here’s the problem – it is not reasonable to expect Verity to be able to perform the tasks outlined above using a straight Garland Drupal website, and it is not a reasonably approximation to the reality of this end user.

Garland looks far to much like Drupal and far to little like a contemporary and designed website. It introduces a raft of challenges to understanding the interface that Verity would never have to address in using the Seven admin theme over the top of a professionally designed website.

We need a theme for testing, however simple, that looks like a proper, designed website. And, in the case of Verity, we need a little structure and content already in place on the website.

Now, there are dozens and dozens of Drupalcamp websites out there… is there some way we can get one of those approximated in Drupal 7 in a reasonably short timeframe?

Ideally, Jeremy will have a theme other than Garland as his default Drupal 7 theme also. But, for me, this is less problematic compared to Verity.


Reporting Findings & Making Decisions

Our testing needs to be video recorded (screencast) – I’ll be using Silverback (because it’s cheap and I use a Mac) but there are many other solutions out there. In order for our findings and recommendations to be agreed and accepted it is important that we’re all able to check back to the source.

There is a Drupal UX Group on Vimeo (http://vimeo.com/groups/drupalux) which seems like the logical place to post these videos (to avoid the 10 minute rule on YouTube if nothing else) and it also makes sense to utilise the existing drupalusability.org infrastructure for reporting findings although it will need to be reworked to allow for the ‘crowdsourced’ and iterative nature of this test approach.

Next Steps

Assuming we agree that this is a good way forward, then there are three key tasks that have to be completed:

a) we need to finalise scripts and make them available to people who wish to conduct testing
b) we need to resolve the issue re: a testing theme, and
c) we need to ensure that drupalusability.org is ready to receive feedback from multiple sources

I know that within the UK there are at least two groups who are keen to make a start on doing some testing before the end of September. It would be great to have this framework in place so that we can make the best possible use of their time and expertise, so, time (as ever) is of the essence.

What do you think? Sound good? Can you help?!

Economist/Drupal – Sprint 2 Demo (CRUD-in-place)

Drupal/Economist Project – Sprint 2 Demo from Leisa on Vimeo.

Today is Demo Day at The Economist, where all the various SCRUM teams will show and tell what they’ve achieved in their latest iteration. I thought I might see if I can get into the habit of pre-recording my demo so I can share it with you here and ask for feedback & advice! So, here we go!

In the past two iterations (this project runs on 1 week iterations) we’ve done a combination of research, design and testing the publishing tools that the editorials staff will use to administer what is known as the ‘Channel Index Pages’ – these are pages that you’d land on if you clicked something in the navigation that said, say, Business & Finance, or Science & Technology, and their job is to pull the most interesting current content to the top so that readers can access it more easily.

These pages are made up of a range of elements, most importantly for this demo are News Packages (consisting of a lead story and related stories/media files) and what is known as the Bloggy Chunk (an editable text area that section editors can freely input their own content, it is still very much a work in progress and is variously used as an aggregator for recent interesting stories that The Economist isn’t covering in depth or to highlight interesting reader comments, it may in the future show content from an RSS feed).

Although working within the SCRUM methodology, I am trying to take a systemwide approach to the design as far as possible, so there has been a whole range of questions that I’ve had to consider that aren’t strictly in the ‘stories’ for our sprint (logging on, activating editing, for example), and similarly, I’ve tried to devise interaction approaches that will be reusable across the other parts of the system that we have yet to design rather than just customising the design approach to this individual problem set.

The design approach shown in this demo is still quite lo-fidelity and ‘broad brushstroke’, it will become much more precise over time, at the moment my priority is to try to work through and communicate the ideas as quickly as possible – giving me more time to explore a range of approaches, and to iterate approaches which has been very valuable.

So, if you have a moment and are so inclined, I’d really love to get your thoughts on the approach we’re taking so far. We’re also about to embark on starting some development work on this – to make sure we can build it in Drupal without too much difficulty – so if anyone has any guidance on how best to approach this, modules we should take a look at, anything else you think we might need to know, I’d love if you could share it here!

Look forward to hearing from you and thanks in advance!

(apologies to anyone who saw the first version of this blogpost with the screwy audio synching)

Economist/Drupal – Design Principles for CRUD-In-Place

I’m going to stop referring to what we’re doing as ‘edit in place’ because actually what we’re attempting to do is what is charmingly known as ‘CRUD’ in Content Management System (CMS) lingo, being Creating, Reading, Updating and Deleting content.

I’ve spent this morning reviewing a whole host of existing examples of ‘CRUD-in-place’ (thanks to the great examples that you provided in response to my previous post). I have to say, although there are quite a few examples out there, not very many of them are particularly inspiring… at least for my purposes.

I’ve posted up some screenshots and a few comments on Flickr that you’re welcome to add your thoughts to. As a summary I think that for this project we can absolutely draw the most inspiration from Flickr & 37 Signals (eg. BaseCamp) implementation of edit in place – mostly for their simplicity, which is the entire point of this exercise.

Many of the other examples offered huge amounts of flexibility which is, frankly, the complete opposite to what I want to offer here – we want to offer flexibility within a template (or perhaps templates) certainly not the ability to create templates on the fly (we don’t want to make people whose skills are editorial suddenly have to try to make good design/user experience decisions!).

Two other pet peeves that I’ve developed since I’ve been reviewing these interfaces are the overly liberal use of buttons when a simple text link would suffice, and throwing up a great hunk of a form that was obviously intended for a back end interface when a much more elegant ‘in place’ solution could have been achieved.

Now that I am trying to apply my learnings and inspiration to my particular design challenge I’m finding myself coming up with some design principles that I will probably add to over time and hope to use consistently throughout the system design. I’m going to start posting them here so I can keep them together and you can give me some feedback as we go!

Design Principles for CRUD-in-place

  1. Don’t map the interface to the technical architecture, map the interface to the tasks that users will want to perform. The user/interface shouldn’t have to care about the way that content is grouped in the back end. If I want to just edit the headline of a ‘news package’ I should be able to do that without having to see the form for the entire package. (Eg. fixing a typo in the headline, adjusting line length etc.)
  2. Show as few controls as possible and try to show them only when required. Particularly on the ‘index’ type pages I’m working on at the moment there will be lots of ‘bits’ that will be editable. Showing all of them will result in a hideously scrappy looking page. I’m rather tempted to use the BaseCamp approach and show the appropriate controls on hover, however I’m also concerned that we design an accessible experience, so we’ll need to think more about what we need to do to achieve this principle without sacrificing the experience for people with visual/mobility impairment.
  3. Where possible, choose a text link and not a button. See (2) above – text links are a much less intrusive element on the page and will help us to maintain a clean, simple interface. Obviously, there are some times where buttons are necessary, we won’t get silly with this.
  4. Provide Editorial Tools not Design Tools. The purpose of this interface is to allow editorial staff (writers and editors) to do their jobs well – their job is to craft, curate and communicate content to readers. These people are not designers and should not be asked to make design decisions. The flexibility they require is about selecting, grouping and prioritising content. The design work should be done within the page template and within that framework we provide appropriate editorial tools.

More to follow I’m sure, I’m just getting started. Feedback welcome!

Economist/Drupal – Complex Editing In Place

Edit In Place

Following on from our work in Sprint 1, I’ve been conducting some research today to establish whether or not we’re going to continue down essentially a traditional ‘admin’ system path or whether we’re going to pursue the altogether more challenging (read: scary & hard work!) route of trying to do as much of the content creation and editing ‘in place’ (which effectively means in as much of a WYSWIG environment as possible).

The research today involved some simple walk/talk throughs of two very basic prototypes that I had made using Omnigraffle and showed in the ‘Presentation’ view (probably the fastest way I’ve found to build relatively complex application interfaces using a mix of screengrabs and wireframes that you can click through). For those who are interested, those and this blog post are going to form my ‘demo’ materials for this Scrum sprint.

Predictably (although also happily) everyones main priority is for things to be ‘as simple as possible’, however there was a strong leaning towards the ‘in place’ solution which I found somewhat surprising, as previous research had almost indicated the opposite. (Yet another reason to research as iteratively as you design!)

So, for the next few days I’m going be working on more detailed interface designs for this ‘edit in place’ approach, and to that end, I wonder if you can help me… I’d *love* to see some good examples of relatively complex edit-in-place/WYSIWYG interfaces. Obviously Flickr is the poster child for fairly simple in-place interactions, but are there good examples of being able to add and create new content elements, move them around on page, maybe even manage a few versions of the page… relatively complicated stuff.

If you could point me to any examples, I’d be incredibly grateful!