in customer research

User research for Drupal.org redesign – what we’ve done, what we’re doing

If you’re following the Drupal.org redesign project, you will no doubt have noticed that we’ve been busy getting to know the Drupal community. It’s been fun so far, and we look forward to doing much more.

I thought you might enjoy a little heads up of what we’ve done and what our plans are (keeping in mind, of course, that a project quite like this has never been planned before, so we reserve the right to switch course at any point in the future if another course looks better!)

‘Lies, damn lies and statistics’

Now, as you can probably tell, we place a lot more stock in talking to the community than poring over numbers (‘what’ is interesting, ‘why’ is much more useful!), but we’re looking at whatever statistical information is being thrown our way (none of it lies, I’m sure! I just really like that quote)

Here’s a little sample you may or may not be familiar with:

Drupal.org has 300,000+ users and gains on average 300 new users per day. There are 375,000 discussion threads between the forums, mailings lists, groups, etc. Our community is active with a 5:1 response rate on general discussions, 4:1 response rate on support discussions and 10:1 response rate on development discussions. Our community is large, engaged and growing. (excerpt from the Redesign Project RFP)

We’re looking at search terms, paths through the site, and other bits and pieces that are somewhat interesting. But nowhere near as interesting as talking… so, onto that part.

Drupalcon Interviews

Mark and I were fortunate to be invited to attend Drupalcon in Szeged, Hungary a couple of weeks ago and we took the opportunity to meet some of the Drupal community whilst we were there. I think we ended up conducting about a dozen depth interviews, between 30-60mins each, with a wide range of participants from a well known ‘maintainer’, through experienced and less experienced Drupal developers, to complete newbies, to people who sell Drupal solutions to their clients – and people from all over the world!

During these interviews we talked a lot about each persons experience with Drupal, with Drupal.org, with the Drupal Community – online and in ‘real’ life. I also asked people to show me a little of how they used Drupal.org which was very enlightening – allowing me to better understand what are the really important bits of Drupal.org for different kinds of users, and which ones of these are all but hidden from the uninitiated!

This was an incredibly worthwhile exercise and really fast-tracked us into understanding both the key issues with the drupal.org website as well as the dynamics of the Drupal community and the different types of experiences that people have of Drupal.org depending on where they sit on the ‘Drupal Learning Curve’.

Recruiting for research participants – the survey

No doubt you’re aware that we’ve had a ‘survey’ running over the past week or so – the purpose of this is to get in touch with people who are willing to help us out with some more structured feedback once we start getting into the design phase and we have a prototype that we’ll be iterating very regularly (at least weekly is the current plan).

If you’ve completed the form, you *may* get an email from us in the next few months asking for your help – this will involve us taking about 45 mins of your time to have a virtual interview – we’ll be using some screen sharing tools so that you can take a look at the prototype, and we can watch how you’re using it, and we’ll have a bit of a chat, ask you some questions etc. as well. It will be fun, and it will also mean that we won’t waste piles of time designing something that isn’t going to work well for the community OR for people who are new to Drupal because – happily, we have recruited lots of people who know little or nothing about Drupal to help us with this exercise – hurrah!

Want to know more about who responded? Here’s some snippets (apologies for the cut off text on the graphs, this is what Google is giving me and I don’t have time/inclination to remake them!):

1024 people have completed the survey so far. Respondents represent  78 different countries and speak about 56 different languages!! (Not including C++, en-au, English (UK), Java, PHP, E Bonics which some people offered as their native language – we take your point!)

10% of respondents don’t know Drupal at all. We’re looking forward to meeting them! Looking after these guys is one area we really need to do better with on drupal.org

Of course, we also need to look after the existing community well – and there are lots of opportunities for improvement – fortunately we have 540 people from the Drupal community who have already put their hands up to help out.

My favourite thing about this chart though is the 359 people who answered ‘not yet!’ when asked if they were a part of the Drupal community. It might not be particularly scientific, but it really does give a sense of the potential that we’re working with here!

Personally, I was quite surprised with the spread of ‘roles’ that people said they were responsible for. Most respondents selected more than one ‘role’ in this survey and that maps with the interviews we did when we were at Drupalcon – for this reason, we’re leaning towards the view that ‘role’ isn’t really a very good way to break down the different types of people we’re designing for (into personas, perhaps) and we’re leaning more towards a breakdown based on ‘closeness’ to the community. More on that soon.

And what are people doing here?

12% seeking ‘overview’ information about Drupal and what it can do
5% looking to be ‘sold’ on Drupal
28% looking to use Drupal to build a website (introductory information, tutorials, modules, themes etc.)
9% comparing Drupal with other CMS options
20% looking for API-level details about Drupal as a content management framework
21% looking for themes, looking for information on how to make Drupal beautiful
40% looking for an answer to a particular problem I’m having with Drupal
51% looking for Drupal modules
22% participating in the Drupal community – I’m a regular
22% exploring the Drupal community – I’m a lurker
30% checking up on Drupal news
9% other (which was frequently to do with checking on projects, issues etc.)

Other channels

We’re also live on Twitter (twitter.com/drupalredesign), Flickr (flickr.com/groups/drupalredesign/) and Slideshare (slideshare.net/group/drupalorg-redesign-project) and taking feedback, suggestions, inspiration – whatever you can throw at us.

Thank you!

I want to take this opportunity to thank everyone who has participated so far – even after such a short time, I really feel as though we know so much more about this project, its rather long history, and some of the background to issues than we would without your input.

We have to work very quickly on this project, so we don’t have a lot of time to labour over all of this information, but we will continue to work openly so if you don’t see us paying attention in the work that we’re putting back to you, there are lots of opportunities to set us straight!

I look forward to working with you more!

  1. Leisa, thank you so much for stepping up to the plate and contributing. New efforts like yours and Kika’s, and all the folks involved with usability testing makes me feel confident that Drupal is moving in the right direction in terms of the end user experience and the administrator experience. As a long-time Drupal user who cares about UX, and feels guilty for not being able to participate in recent years, I’m glad that UX people are going to be around to insert some rigor into the process of ensuring that the public site and the software experience are going in the right direction. Kudos.

  2. Well, That’s quite an impressive work. thanks for sharing it. by the way ,i filled survay form as well, and that was fun :))

  3. Hi Leisa, I’m not a Drupal user, I’m following along with great interest from a UX perspective. I’d be really interested in knowing more about the prototype you built for testing. How much detail did you go into, how deep did it go (and how did you deal with users trying to go beyond the boundries of what was prototypes) and how did you build it (was it HTML, with some JS, Flash, or whatever). Is there any chance of getting a look at it?

  4. thanks everyone for the encouragement!
    @Dan – there is no prototype yet – we’re not up to that part yet! – but as it comes into existence, it will no doubt be available for you to take a look at.

    You can expect to see some very early ‘paper/sketch’ prototypes coming soon, but we are looking to move into HTML/CSS as quickly as possible.

Comments are closed.