UCD tools

Guerrilla Techniques – Does inexpensive research have to be ‘quick & dirty’?

I’m really interested to hear what guerrilla style techniques you’re using to do User Research when there’s not a lot of budget or if you don’t have the traditional research facilities or infrastructure?

I imagine there’s some pretty interesting stuff going on out there, what with all the new and often free web based tools that we have available that should make observational research more and more accessible to all of us.

More and more of us are using a combination of camera phones and sites like Flickr in place of traditional ‘diary studies’, and some of us have investigated using tools like Twitter for this purpose as well (has anyone actually done a contextual research study using Twitter yet? I’d love to hear about it).

I also hear great stories of more and more people getting out into Starbucks and other public places (although, for some reason, it often seems to be Starbucks – wifi I guess) and doing some usability testing with unsuspecting members of the public.

Do you have strategies for inexpensive and rapid recruitment techniques that actually allow you to recruit to a profile? (Or multiple profiles). Could social networking tools like FaceBook or Twitter (again) play a role here perhaps? (Insert concerns re: bias in audience sample)

Has anyone come up with a video set up that allows for both screen capture and a video of the user without needing two computers? (I have a webcam built into my MacBook… surely it’s feasible!)

What other wild and wacky – but most of all inexpensive and accessible – techniques and tools are you using to find out more about the people who use (or might use) whatever it is you’re designing?

Are we getting to the point where, perhaps, we can do better research outside of the lab than inside it?

That’s a whole lotta questions. What say you?

14 thoughts on “Guerrilla Techniques – Does inexpensive research have to be ‘quick & dirty’?

  1. Hi Leisa, If you want to try something like this in the Sydney CBD and you want an extra set of hands, let me know. I’d be very interested in extrapolating any experiences from this form of research to quick ‘n’ dirty PR/comms/marketing research.

  2. Leisa,
    Great topic and I hope to hear more from this as well. I gotta say that I did some inpromptu evaluations with users on some wireframes/prototypes my group developed for our school project. It was actually a lot of fun and well received especially since we were in context to our design problem (grocery store). What I quickly learned was that there is a lot of travel necessary because location specific testing may skew data dude to the kinds of people that populate the area. Getting a proper sample of the target audience is tough, so dividing up the effort is important.

  3. Super ideas, Leisa! I especially like the cam+screen idea – there MUST be something, surely.

    As for the recruiting issue, I think your Starbucks/Twitter/Facebook, etc. approach is useful for any project, but your concerns over skewed sample are valid.

    As an example, I did some testing last week with a group going in fairly cold to an off-the-shelf web app and testing three relatively simple scenarios (or so I thought). The group were all computer familiar and use several different apps each day, yet just one of five was able to complete the scenarios. I think online recruitment will get you a larger success group.

    Oh, and old-school business and government clients might also get antsy over the recruitment approach given their fear of exposure.

  4. I would also be very interested to know what is going on in this area. I am about to commission some ‘low cost as possible’ user research on behalf of a govt organisation. My thinking so far included the blogging community and I have been looking into Twitter as well. However the comments about skewing results though testing an online-literate community are very valid.
    Good topic!

  5. Dunno about the Mac, Leisa, but on the PC Camtasia’s an excellent and cheap (£160?) tool for recording screen, audio and webcam video. . I’d say it’s worth getting hold of a ‘quick and dirty’ Windows laptop or a copy of Parallels to run it.

    As for doing more out of the lab than in it, definitely. (I’ve never been a fan of big, expensive labs anyway. They can often be too formal and intimidating for participants. That said, if you’ve got the money they can be done really well. Nokia in Huntingdon have a fantastic lab that mimics a posh sitting room.) I’m finding ‘cafe’ usability sessions are fast replacing lab sessions for the ‘standard’ evaluation work I do. You can run them at all sorts of places – trade shows, staff canteens, conferences, schools – and as Kevin points out, you’re often in the context of use. Combined with the ‘listening lab’ approach (in a nutshell: ‘what are you interested in using this kind of site for, now try and use it for the things you’ve just told me’) it’s a really effective way of getting a user-centred rather than usability evaluator-centred insight.

    I’m just starting out with remote tools and I think that I’d actually use remote card sorting in preference to face-to-face sessions. What’s exciting for me is the access to large sample sizes and the statistical reliability that comes with that. As for flickr diary studies, I’d argue that there’s still a lot you can do with ‘traditional’ diary studies. (I blogged about some digital vs traditional diaries last month ). And they still come up pretty cheap

  6. I’ll definitely be watching to see what others are adding to the discussion, as we’re in the “low budget” group. Unfortunately, we just found out legal isn’t too keen on us doing this kind of research so my plans of snagging impromptu participants at the mall and community center for an upcoming test are nixed.

    In the past, I’ve had good luck turning to specific user groups (both real-life and virtual communities) when we’re looking at a specific target population. For example, going to a local Mac users group meeting or scoping out online food discussion forums for people who prefer to buy organic foods.

    Also, regarding the single computer for camera & screen recording, it can be done. Techsmith (makers of Camtasia) make a program called Morae that does simultaneous capture of on-screen events (and logs keystrokes, clicks, etc.) and video of the user from a USB webcam. If you want to mark participant actions/timepoints, that can be done remotely via another machine. Unfortunately it only works for PCs right now- however, at CHI I did see it running on a Macbook with the built-in webcam via Bootcamp; but I’m not 100% sure that’s officially supported. If I’ve done my tags right, you should be able to go here for more info: http://www.techsmith.com/morae.asp

  7. Very interesting post Leisa, I’d really like to try out some of the things you’ve mentioned.
    I’ve previously used Gumtree http://www.gumtree.com/ to recruit participants on a (no!) budget project. We were specifically working with members of the Latin American community in London and this site was an excellent tool to reach this very specific target audience.
    Paper-prototyping is cheap & effective – For the project mentioned above I paper prototyped a mobile social software app – here’s a link to a project report with more details- http://tinyurl.com/2f3994 (PDF)
    I’ve used Morae, which is very good for screen recording, video and audio, but for a no-budget option a free screen recorder with audio such as Camstudio – http://www.camstudio.org/ is the next-best.
    Ultimately, I think the best-value tool is your imagination!

  8. I’m a great believer also in paper prototyping. If nothing else, it gets the bugs sorted out of the business process and it’s cheap, cheap, cheap…I also tend to use photocopies of actual sketches of wireframes rather than nicely processed ones because a) it’s quicker and b) people seem more at ease with starting to scribble on them themselves than on some beautifully produced story board or other.

  9. Hi Leisa
    We are using a method we call mobile diaries – which takes advantage of mobile phones as in situ reporting devices, and blogs as a place to collect, reflect and comment on those reports/posts. We find them to be a very rich, informative and collaborative research method, straight forward to deploy and cost effective. That said (instead of using some of the existing free media sharing sites) we did need to customise a word press blog in order to retain the rights to our (participants) content, and to support the range of media (audio, text, video and image) formats we wanted.

    We also do “beer and pizza” nights (which does include other non alcoholic beverages of course) nights where we do informal usability testing in house, as well as giving participants lofi video cameras to “tell their own story” of using the site, which generates a different kind of feedback.

    I am also interested in the increasing number of research companies using online tools such as blogs (generally customised) to collect research data and feedback such as http://www.kdaresearch.com/blog/?p=30

    I look forward to other posts.

  10. What an interesting topic! As an independent research designer/strategist, I try to be in the field as much as possible and archive the key findings and thoughts to see if any of the discoveries come useful when I work on the future project.

    As far as the method and tools…I would love to see the opportunity to leverage some of the social network sites like twitters to learn about consumers.

    Looking forward to your post!

  11. Hi, regarding some research based on twitter maybe this pdf is interesting for you because of the approach when using twitter usage statistics for research.

    Why We Twitter: Understanding Microblogging Usage and Communities

Comments are closed.