User research for redesign – what we’ve done, what we’re doing

If you’re following the redesign project, you will no doubt have noticed that we’ve been busy getting to know the Drupal community. It’s been fun so far, and we look forward to doing much more.

I thought you might enjoy a little heads up of what we’ve done and what our plans are (keeping in mind, of course, that a project quite like this has never been planned before, so we reserve the right to switch course at any point in the future if another course looks better!)

‘Lies, damn lies and statistics’

Now, as you can probably tell, we place a lot more stock in talking to the community than poring over numbers (‘what’ is interesting, ‘why’ is much more useful!), but we’re looking at whatever statistical information is being thrown our way (none of it lies, I’m sure! I just really like that quote)

Here’s a little sample you may or may not be familiar with: has 300,000+ users and gains on average 300 new users per day. There are 375,000 discussion threads between the forums, mailings lists, groups, etc. Our community is active with a 5:1 response rate on general discussions, 4:1 response rate on support discussions and 10:1 response rate on development discussions. Our community is large, engaged and growing. (excerpt from the Redesign Project RFP)

We’re looking at search terms, paths through the site, and other bits and pieces that are somewhat interesting. But nowhere near as interesting as talking… so, onto that part.

Drupalcon Interviews

Mark and I were fortunate to be invited to attend Drupalcon in Szeged, Hungary a couple of weeks ago and we took the opportunity to meet some of the Drupal community whilst we were there. I think we ended up conducting about a dozen depth interviews, between 30-60mins each, with a wide range of participants from a well known ‘maintainer’, through experienced and less experienced Drupal developers, to complete newbies, to people who sell Drupal solutions to their clients – and people from all over the world!

During these interviews we talked a lot about each persons experience with Drupal, with, with the Drupal Community – online and in ‘real’ life. I also asked people to show me a little of how they used which was very enlightening – allowing me to better understand what are the really important bits of for different kinds of users, and which ones of these are all but hidden from the uninitiated!

This was an incredibly worthwhile exercise and really fast-tracked us into understanding both the key issues with the website as well as the dynamics of the Drupal community and the different types of experiences that people have of depending on where they sit on the ‘Drupal Learning Curve’.

Recruiting for research participants – the survey

No doubt you’re aware that we’ve had a ‘survey’ running over the past week or so – the purpose of this is to get in touch with people who are willing to help us out with some more structured feedback once we start getting into the design phase and we have a prototype that we’ll be iterating very regularly (at least weekly is the current plan).

If you’ve completed the form, you *may* get an email from us in the next few months asking for your help – this will involve us taking about 45 mins of your time to have a virtual interview – we’ll be using some screen sharing tools so that you can take a look at the prototype, and we can watch how you’re using it, and we’ll have a bit of a chat, ask you some questions etc. as well. It will be fun, and it will also mean that we won’t waste piles of time designing something that isn’t going to work well for the community OR for people who are new to Drupal because – happily, we have recruited lots of people who know little or nothing about Drupal to help us with this exercise – hurrah!

Want to know more about who responded? Here’s some snippets (apologies for the cut off text on the graphs, this is what Google is giving me and I don’t have time/inclination to remake them!):

1024 people have completed the survey so far. Respondents represent  78 different countries and speak about 56 different languages!! (Not including C++, en-au, English (UK), Java, PHP, E Bonics which some people offered as their native language Рwe take your point!)

10% of respondents don’t know Drupal at all. We’re looking forward to meeting them! Looking after these guys is one area we really need to do better with on

Of course, we also need to look after the existing community well – and there are lots of opportunities for improvement – fortunately we have 540 people from the Drupal community who have already put their hands up to help out.

My favourite thing about this chart though is the 359 people who answered ‘not yet!’ when asked if they were a part of the Drupal community. It might not be particularly scientific, but it really does give a sense of the potential that we’re working with here!

Personally, I was quite surprised with the spread of ‘roles’ that people said they were responsible for. Most respondents selected more than one ‘role’ in this survey and that maps with the interviews we did when we were at Drupalcon – for this reason, we’re leaning towards the view that ‘role’ isn’t really a very good way to break down the different types of people we’re designing for (into personas, perhaps) and we’re leaning more towards a breakdown based on ‘closeness’ to the community. More on that soon.

And what are people doing here?

12% seeking ‘overview’ information about Drupal and what it can do
5% looking to be ‘sold’ on Drupal
28% looking to use Drupal to build a website (introductory information, tutorials, modules, themes etc.)
9% comparing Drupal with other CMS options
20% looking for API-level details about Drupal as a content management framework
21% looking for themes, looking for information on how to make Drupal beautiful
40% looking for an answer to a particular problem I’m having with Drupal
51% looking for Drupal modules
22% participating in the Drupal community – I’m a regular
22% exploring the Drupal community – I’m a lurker
30% checking up on Drupal news
9% other (which was frequently to do with checking on projects, issues etc.)

Other channels

We’re also live on Twitter (, Flickr ( and Slideshare ( and taking feedback, suggestions, inspiration – whatever you can throw at us.

Thank you!

I want to take this opportunity to thank everyone who has participated so far – even after such a short time, I really feel as though we know so much more about this project, its rather long history, and some of the background to issues than we would without your input.

We have to work very quickly on this project, so we don’t have a lot of time to labour over all of this information, but we will continue to work openly so if you don’t see us paying attention in the work that we’re putting back to you, there are lots of opportunities to set us straight!

I look forward to working with you more!

Opening the floodgates! How to participate in the redesign project

Suddenly, there are more ways that you can be involved in the redesign project – do any of these take your fancy? Redesign Twitter Group

If you’re on Twitter (or if you’d like to be), follow the Redesign Twitter Group at

Once you’re following you can share a message with everyone who’s interested in the redesign project by sending a direct message (start the message with a ‘d’) to the group and everyone who is following the group will receive the tweet.

Something on bothering you? Simple ‘d drupalredesign‘ then tell us what’s bothering you (in 140 char or less!). Your messages will be shared with others on the group. Redesign Flickr Group

A picture’s worth a thousand words right? That’s probably at least a few dozen Tweets ;)

If you’re a Flickr member (or you’d like to be) you can join and contribute to the Redesign Flickr Group. The idea is that you can post screenshots of things you think work well that we should pay attention to, or things you think need attention on

A good way to do this is to take a screenshot and add some notes so that we know what you think is good/bad/indifferent – if you’re Mac user, you could consider using (as Dries suggested in his keynote at Drupalcon recently). I’m sure others have similar great non-Mac applications to suggest!

You can also add more notes and comments to screenshots that others have uploaded.

The Redesign Flickr Group can be found here: Redesign Slideshare Group

We’ve set up a group on for the redesign project – any presentations related to the project will be added to this group. (Mark – you’d better put the Drupalcon keynote up when you have a spare moment!) Mark has already put up the Redesign Keynote from Drupalcon. (Thanks Mark!)

You can find it here:

Participate in one-on-one interviews during the design process

Of course, if you haven’t already signed up by filling out our short survey, you can do that too. If you do, we may contact you in the course of the next few months to ask you to review one of the iterations of the prototype we’ll be working on. We’ll have some questions for you to answer and some tasks to do. It’ll be fun! You can sign up for that here:

Of course, there is always the Redesign Group at which you can find and join here ( if you haven’t already!

We really look forward to having you involved in this journey (and beyond!).

Update: for those wondering, I’ve used GroupTweet to set up the Twitter group – I don’t know if this is the best solution or not so more than happy to hear your ideas/suggestions/feedback etc.

I’m still on the hunt for a good solution to auto-follow, so the group can follow you back immediately. Any recommentations?

I’ve signed up to SocialToo for auto-follow on Twitter – hopefully it works well enough!

Why I care what you had for lunch

There is an excellent article, I’m So Totally, Digitally, Close To You, about ambient intimacy in the New York Times Magazine this weekend, and I’m honoured to have been able to contribute a few words via a fascinating chat with the author, Clive Thompson. (This is the second time I’ve gotten my name mentioned in the NYT can you believe! but this is the first time they’ve spelt it correctly).

Unsurprisingly, the comments to this article are scattered with the standard ‘who cares what you had for lunch’, ‘you guys are way too into yourselves’ and ‘you don’t pay enough attention to real life’ type responses. All of which, in their own way, are quite understandable and some of them (particularly the ‘you don’t pay proper attention to your real life’ can often be all too true).

I’ve been thinking quite a bit about why I think these ‘lunch’ type tweets are important. I’ve decided there are at least three reasons why I care what you had for lunch.

They are:

  1. I see your Tweet in my Twitter stream and I think of you. It keeps you present in my life. I like that.
  2. I feel like I’m more present in your life because I know more about the small details – like what you had for lunch – that I wouldn’t know if you didn’t share it with me. It makes me feel closer to you.
  3. I learn more about you (if I didn’t know you well before we started trading Tweets). Perhaps I learn that you’re a vegetarian, or I learn which restaurants are your favourites. I get to know you more.

Twitter means that I stay in a constant state of connectedness with you, in a constant handshake, constantly doing small talk… and this is actually incredibly valuable to me for reasons that go *way* beyond touchy feely.

How so?

Well, I am in the fortunate situation of being connected to quite a few very clever people on Twitter, and by staying in this perpetual handshake with them, it means that when I *need* something from them, I’m able to put out the question without having to do that small talk dance that we usually need to do.

You know when you need to ask for a favour? First you need to identify the right person to ask, then contact them, do some polite small talk and catching up, then finally, to the favour, which they may or may not be able to help you with.

Compare to Twitter – I have a question/problem I need solved – I put it out to my Twitter network, and – often within moments – I have answers, problems solved.

Similarly, I’ll answer questions and hopefully help solve problems for others as well.

Despite the fact that I’m listening (reading, really) what you and many others had for lunch, Twitter is still an incredibly efficient way to problem solve, it’s a remarkably rapid and rich resource. Far beyond anything I’ve experienced in the past. For example, as a freelancer, Twitter is now my IT help desk, and I’ve never had one better.

I do love the touchy feely ‘friendship’ side of Twitter – the ‘intimacy’ aspect is very valuable to me (although I do also love seeing my friends in real life whenever it is possible!), but the thing that continues to make my jaw drop with Twitter is the access that I have to distributed expertise and opinion. I use it regularly in my personal and professional life now and hope never to be without it.

Meanwhile. I’m off to have dinner. We’re having pizza tonight ;)

Try Google Docs for survey or recruitment forms

Just a quick note to recommended using Google Docs ‘forms’ as a free tool to manage surveys and recruitment. (Choose New, then Form).

We recently wanted to invite people to participate in user research for the redesign project – as a part of this we had a short screener we wanted to run people through so that we can target research appropriately in the coming months (and also get some interesting stats – more on that soon!).

Initially I was planning to use Ethnio, as it is purpose built for this, looks pretty and has a kind of nice DHTML ‘not-popup’. I couldn’t get it working though, so then turned to the ever trusty Survey Monkey, but… eh, so ugly! At the last minute I thought of Google Docs and that’s where we stayed.

Super easy to set up, and a nice clean looking interface out of the box, plus no worries being charged for having too many responses. Easy peasy.

We have since almost 900 responses in a just few days and it seems to have held up nicely.

So, if you are looking for a nice tool to use as a screener or a questionnaire and you’re not too fussed about customising the look and feel, I’d heartily recommend Google Docs.

Disclaimer, disclaimer etc. I’m sure Ethnio works beautifully for lots of people. I tried to get it working for several days without luck and by the time support got back to me, we had hundreds of survey responses to the Google version. I’m also sure you can make Survey Monkey look grand, but I don’t know how and didn’t want to spend the time finding out.