disambiguity

User research for Drupal.org redesign - what we’ve done, what we’re doing

If you’re following the Drupal.org redesign project, you will no doubt have noticed that we’ve been busy getting to know the Drupal community. It’s been fun so far, and we look forward to doing much more.

I thought you might enjoy a little heads up of what we’ve done and what our plans are (keeping in mind, of course, that a project quite like this has never been planned before, so we reserve the right to switch course at any point in the future if another course looks better!)

‘Lies, damn lies and statistics’

Now, as you can probably tell, we place a lot more stock in talking to the community than poring over numbers (’what’ is interesting, ‘why’ is much more useful!), but we’re looking at whatever statistical information is being thrown our way (none of it lies, I’m sure! I just really like that quote)

Here’s a little sample you may or may not be familiar with:

Drupal.org has 300,000+ users and gains on average 300 new users per day. There are 375,000 discussion threads between the forums, mailings lists, groups, etc. Our community is active with a 5:1 response rate on general discussions, 4:1 response rate on support discussions and 10:1 response rate on development discussions. Our community is large, engaged and growing. (excerpt from the Redesign Project RFP)

We’re looking at search terms, paths through the site, and other bits and pieces that are somewhat interesting. But nowhere near as interesting as talking… so, onto that part.

Drupalcon Interviews

Mark and I were fortunate to be invited to attend Drupalcon in Szeged, Hungary a couple of weeks ago and we took the opportunity to meet some of the Drupal community whilst we were there. I think we ended up conducting about a dozen depth interviews, between 30-60mins each, with a wide range of participants from a well known ‘maintainer’, through experienced and less experienced Drupal developers, to complete newbies, to people who sell Drupal solutions to their clients - and people from all over the world!

During these interviews we talked a lot about each persons experience with Drupal, with Drupal.org, with the Drupal Community - online and in ‘real’ life. I also asked people to show me a little of how they used Drupal.org which was very enlightening - allowing me to better understand what are the really important bits of Drupal.org for different kinds of users, and which ones of these are all but hidden from the uninitiated!

This was an incredibly worthwhile exercise and really fast-tracked us into understanding both the key issues with the drupal.org website as well as the dynamics of the Drupal community and the different types of experiences that people have of Drupal.org depending on where they sit on the ‘Drupal Learning Curve’.

Recruiting for research participants - the survey

No doubt you’re aware that we’ve had a ’survey’ running over the past week or so - the purpose of this is to get in touch with people who are willing to help us out with some more structured feedback once we start getting into the design phase and we have a prototype that we’ll be iterating very regularly (at least weekly is the current plan).

If you’ve completed the form, you *may* get an email from us in the next few months asking for your help - this will involve us taking about 45 mins of your time to have a virtual interview - we’ll be using some screen sharing tools so that you can take a look at the prototype, and we can watch how you’re using it, and we’ll have a bit of a chat, ask you some questions etc. as well. It will be fun, and it will also mean that we won’t waste piles of time designing something that isn’t going to work well for the community OR for people who are new to Drupal because - happily, we have recruited lots of people who know little or nothing about Drupal to help us with this exercise - hurrah!

Want to know more about who responded? Here’s some snippets (apologies for the cut off text on the graphs, this is what Google is giving me and I don’t have time/inclination to remake them!):

1024 people have completed the survey so far. Respondents represent  78 different countries and speak about 56 different languages!! (Not including C++, en-au, English (UK), Java, PHP, E Bonics which some people offered as their native language - we take your point!)

10% of respondents don’t know Drupal at all. We’re looking forward to meeting them! Looking after these guys is one area we really need to do better with on drupal.org

Of course, we also need to look after the existing community well - and there are lots of opportunities for improvement - fortunately we have 540 people from the Drupal community who have already put their hands up to help out.

My favourite thing about this chart though is the 359 people who answered ‘not yet!’ when asked if they were a part of the Drupal community. It might not be particularly scientific, but it really does give a sense of the potential that we’re working with here!

Personally, I was quite surprised with the spread of ‘roles’ that people said they were responsible for. Most respondents selected more than one ‘role’ in this survey and that maps with the interviews we did when we were at Drupalcon - for this reason, we’re leaning towards the view that ‘role’ isn’t really a very good way to break down the different types of people we’re designing for (into personas, perhaps) and we’re leaning more towards a breakdown based on ‘closeness’ to the community. More on that soon.

And what are people doing here?

12% seeking ‘overview’ information about Drupal and what it can do
5% looking to be ’sold’ on Drupal
28% looking to use Drupal to build a website (introductory information, tutorials, modules, themes etc.)
9% comparing Drupal with other CMS options
20% looking for API-level details about Drupal as a content management framework
21% looking for themes, looking for information on how to make Drupal beautiful
40% looking for an answer to a particular problem I’m having with Drupal
51% looking for Drupal modules
22% participating in the Drupal community - I’m a regular
22% exploring the Drupal community - I’m a lurker
30% checking up on Drupal news
9% other (which was frequently to do with checking on projects, issues etc.)

Other channels

We’re also live on Twitter (twitter.com/drupalredesign), Flickr (flickr.com/groups/drupalredesign/) and Slideshare (slideshare.net/group/drupalorg-redesign-project) and taking feedback, suggestions, inspiration - whatever you can throw at us.

Thank you!

I want to take this opportunity to thank everyone who has participated so far - even after such a short time, I really feel as though we know so much more about this project, its rather long history, and some of the background to issues than we would without your input.

We have to work very quickly on this project, so we don’t have a lot of time to labour over all of this information, but we will continue to work openly so if you don’t see us paying attention in the work that we’re putting back to you, there are lots of opportunities to set us straight!

I look forward to working with you more!

Opening the floodgates! How to participate in the Drupal.org redesign project

Suddenly, there are more ways that you can be involved in the drupal.org redesign project - do any of these take your fancy?

Drupal.org Redesign Twitter Group

If you’re on Twitter (or if you’d like to be), follow the Drupal.org Redesign Twitter Group at www.twitter.com/drupalredesign

Once you’re following you can share a message with everyone who’s interested in the redesign project by sending a direct message (start the message with a ‘d’) to the group and everyone who is following the group will receive the tweet.

Something on Drupal.org bothering you? Simple ‘d drupalredesign‘ then tell us what’s bothering you (in 140 char or less!). Your messages will be shared with others on the group.

Drupal.org Redesign Flickr Group

A picture’s worth a thousand words right? That’s probably at least a few dozen Tweets

If you’re a Flickr member (or you’d like to be) you can join and contribute to the Drupal.org Redesign Flickr Group. The idea is that you can post screenshots of things you think work well that we should pay attention to, or things you think need attention on Drupal.org

A good way to do this is to take a screenshot and add some notes so that we know what you think is good/bad/indifferent - if you’re Mac user, you could consider using Skitch.com (as Dries suggested in his keynote at Drupalcon recently). I’m sure others have similar great non-Mac applications to suggest!

You can also add more notes and comments to screenshots that others have uploaded.

The Drupal.org Redesign Flickr Group can be found here: http://flickr.com/groups/drupalredesign/

Drupal.org Redesign Slideshare Group

We’ve set up a group on Slideshare.net for the redesign project - any presentations related to the project will be added to this group. (Mark - you’d better put the Drupalcon keynote up when you have a spare moment!) Mark has already put up the Redesign Keynote from Drupalcon. (Thanks Mark!)

You can find it here: http://www.slideshare.net/group/drupalorg-redesign-project

Participate in one-on-one interviews during the design process

Of course, if you haven’t already signed up by filling out our short survey, you can do that too. If you do, we may contact you in the course of the next few months to ask you to review one of the iterations of the prototype we’ll be working on. We’ll have some questions for you to answer and some tasks to do. It’ll be fun! You can sign up for that here: http://icanhaz.com/drupalresearch

Of course, there is always the Redesign Group at Drupal.org which you can find and join here (http://groups.drupal.org/drupal-org-redesign-analysis) if you haven’t already!

We really look forward to having you involved in this journey (and beyond!).

Update: for those wondering, I’ve used GroupTweet to set up the Twitter group - I don’t know if this is the best solution or not so more than happy to hear your ideas/suggestions/feedback etc.

I’m still on the hunt for a good solution to auto-follow, so the group can follow you back immediately. Any recommentations?

I’ve signed up to SocialToo for auto-follow on Twitter - hopefully it works well enough!

Why I care what you had for lunch

There is an excellent article, I’m So Totally, Digitally, Close To You, about ambient intimacy in the New York Times Magazine this weekend, and I’m honoured to have been able to contribute a few words via a fascinating chat with the author, Clive Thompson. (This is the second time I’ve gotten my name mentioned in the NYT can you believe! but this is the first time they’ve spelt it correctly).

Unsurprisingly, the comments to this article are scattered with the standard ‘who cares what you had for lunch’, ‘you guys are way too into yourselves’ and ‘you don’t pay enough attention to real life’ type responses. All of which, in their own way, are quite understandable and some of them (particularly the ‘you don’t pay proper attention to your real life’ can often be all too true).

I’ve been thinking quite a bit about why I think these ‘lunch’ type tweets are important. I’ve decided there are at least three reasons why I care what you had for lunch.

They are:

1. I see your Tweet in my Twitter stream and I think of you. It keeps you present in my life. I like that.
2. I feel like I’m more present in your life because I know more about the small details - like what you had for lunch - that I wouldn’t know if you didn’t share it with me. It makes me feel closer to you.
3. I learn more about you (if I didn’t know you well before we started trading Tweets). Perhaps I learn that you’re a vegetarian, or I learn which restaurants are your favourites. I get to know you more.

Twitter means that I stay in a constant state of connectedness with you, in a constant handshake, constantly doing small talk… and this is actually incredibly valuable to me for reasons that go *way* beyond touchy feely.

How so?

Well, I am in the fortunate situation of being connected to quite a few very clever people on Twitter, and by staying in this perpetual handshake with them, it means that when I *need* something from them, I’m able to put out the question without having to do that small talk dance that we usually need to do.

You know when you need to ask for a favour? First you need to identify the right person to ask, then contact them, do some polite small talk and catching up, then finally, to the favour, which they may or may not be able to help you with.

Compare to Twitter - I have a question/problem I need solved - I put it out to my Twitter network, and - often within moments - I have answers, problems solved.

Similarly, I’ll answer questions and hopefully help solve problems for others as well.

Despite the fact that I’m listening (reading, really) what you and many others had for lunch, Twitter is still an incredibly efficient way to problem solve, it’s a remarkably rapid and rich resource. Far beyond anything I’ve experienced in the past. For example, as a freelancer, Twitter is now my IT help desk, and I’ve never had one better.

I do love the touchy feely ‘friendship’ side of Twitter - the ‘intimacy’ aspect is very valuable to me (although I do also love seeing my friends in real life whenever it is possible!), but the thing that continues to make my jaw drop with Twitter is the access that I have to distributed expertise and opinion. I use it regularly in my personal and professional life now and hope never to be without it.

Meanwhile. I’m off to have dinner. We’re having pizza tonight

Try Google Docs for survey or recruitment forms

Just a quick note to recommended using Google Docs ‘forms’ as a free tool to manage surveys and recruitment. (Choose New, then Form).

We recently wanted to invite people to participate in user research for the drupal.org redesign project - as a part of this we had a short screener we wanted to run people through so that we can target research appropriately in the coming months (and also get some interesting stats - more on that soon!).

Initially I was planning to use Ethnio, as it is purpose built for this, looks pretty and has a kind of nice DHTML ‘not-popup’. I couldn’t get it working though, so then turned to the ever trusty Survey Monkey, but… eh, so ugly! At the last minute I thought of Google Docs and that’s where we stayed.

Super easy to set up, and a nice clean looking interface out of the box, plus no worries being charged for having too many responses. Easy peasy.

We have since almost 900 responses in a just few days and it seems to have held up nicely.

So, if you are looking for a nice tool to use as a screener or a questionnaire and you’re not too fussed about customising the look and feel, I’d heartily recommend Google Docs.

Disclaimer, disclaimer etc. I’m sure Ethnio works beautifully for lots of people. I tried to get it working for several days without luck and by the time support got back to me, we had hundreds of survey responses to the Google version. I’m also sure you can make Survey Monkey look grand, but I don’t know how and didn’t want to spend the time finding out.

Contemplating Open Source UX

There are many reasons why I am tremendously excited about being involved in the redesign project for Drupal.org, not the least of which is that - for a change - I’m actually allowed to talk to you all about the project as we go. Afterall, it’s an open source project - we don’t care so much for confidentiality and Intellectual Property, what we care about here is being open and being part of the community.

Exciting yes? but also somewhat terrifying! What an amazing (and enormous!) community to try to become a part of! As Mark said in our keynote at Drupalcon - it feels like being the new kid at school - will we make friends?!  But one of the things I’m really thinking hard about is how to harness their amazingness in the best way for this project?

Back in the olden days I was a project manager, so I have a great appreciation for an approach that works to limit the amount of feedback that you take into a project - how much, how often etc. Trying to get ‘consolidated’ feedback has traditionally been the goal, so that we can move through the design process as efficiently and calmly as possible.

As a part of our plan we have already factored in ‘community feedback’ to the iterations of the prototype that we’ll be releasing from very early in the design lifecycle - the community *will* be involved in this project, albeit in a somewhat structured way.

But, against all of my project management instincts, I am itching to get as much community involvement in this project as I possibly can! To encourage the entire community to think about things like experience strategies, and information architecture and  user centred design.

I am tempted to set up a Twitter group (@drupalredesign perhaps) where we can all tweet little brainbursts we have about the redesign. To set up a Flickr group where we can all post annotated screenshots of stuff we like, stuff that’s broken. To blog about half finished ideas I’m having about strategies and solutions we’re working on.

This is all ridiculously dangerous from a expectation management perspective - there is no way on earth that we could make any assurances about taking everything into consideration, answering every suggestion or issue raised, solving all of the problems…. (don’t let me start coining a project management buzzword that involves ambiance!)

And yet… it’s also ridiculously exciting.

Already, from blogging about the gaming / karma issue, I’m now talking to someone out in Drupal land who is already working on a module we might be able to use. I wonder whether we would have made that connection otherwise.

Eh. I almost feel as though it is inevitable. We have to open up completely, and just see what happens.

I’m almost certain some expectation management/ community disasters will ensue, but hopefully also some amazingness. I’m sure you’ll hear all about it as we go!

What say you? Am I having a moment of insanity? Or shall we open the floodgates and see what happens?

Research opportunity - ‘Proud Mums’ and ‘Good Friends’ earn £40

I’m looking for some people to partipate in a research project in or around Central London next week (9-10 September). More information here.

Social literacy - does ‘karma gaming’ matter?

I have a question for all of you smart people out there. Does the fact that some people ‘game’ karma or rankings or anything you can count in social networks matter?

Here’s the context - I’m doing some work with the Drupal crew and at first blush it seems (perhaps obviously?) that Drupal.org is actually much more of a ’social’ site than a ‘content’ site, and that many of the social and content issues we need to address might be helped along by making the activity that community members are undertaking visible - which essentially becomes a kind of ‘karma’ (a la Slashdot)

Karma is the sum of your activity on Slashdot. This means posting, moderation, story submissions. It’s just an integer in a database. The tiers are Terrible, Bad, Neutral, Positive, Good, and Excellent.

The obvious objection to this is gaming - that people will behave in a way that will increase their karma which is not necessarily in the best interest of the community at large.

There are a number of ways that you can design karma systems so that they are less likely to be gamed (see again Slashdot):

People like to treat their Slashdot Karma like some sort of video game, with a numeric integer representing their score in the game. People who do this simply are missing the point. The text label is one way we’ve decided to emphasize the point that karma doesn’t matter.

and

Yes. Karma is now capped at “Excellent” This was done to keep people from running up insane karma scores, and then being immune from moderation. Despite some theories to the contrary, the karma cap applies to every account.

And, of course, the well documented removal of the ‘top diggers’ page from Digg is another example of removing the incentive to game karma.

Help redesign drupal.org