Comments on: Embracing the Un-Science of Qualitative Research Part Two – Ever-Evolving Prototypes are Ace

By: Kristen

Kristen — Fri, 03 Aug 2007 19:02:46 +0000

I agree with this 3 part series and look forward to the last part. But my question is, what about generalizations? Having rich data from a few participants is great, but what works for one doesn’t always work for another. Can you then make generalizations? And if so, how can you be sure you’re adequately representing the participant population? And, in a formal environment, how can you justify your generalizations?

By: Embracing the Un-Science of Qualitative Research « ilab

Embracing the Un-Science of Qualitative Research « ilab — Sun, 29 Jul 2007 21:25:17 +0000

[...] Embracing the Un-Science of Qualitative Research Part Two – Ever-Evolving Prototypes are Ace One of the fundamentals of quantitative research is its systematic nature. It’s about measuring stuff. And, you don’t want that stuff to change as you’re measuring it for a number of reasons—not the least of which being that it makes it very difficult to plot on a graph [...]

By: Qualitative Research » UIE Brain Sparks

Qualitative Research » UIE Brain Sparks — Wed, 25 Jul 2007 17:05:22 +0000

[...] You can read part 2 here: Embracing the Un-Science of Qualitative Research Part Two – Ever-Evolving Prototypes are Ace Part 3 is on the way. [...]

By: Michael Clarke

Michael Clarke — Tue, 17 Jul 2007 15:45:24 +0000

Ever-evolving prototype – I do like that. And if you’re trying to hit a tough deadline, it’s the only way to go.

By: Lance

Lance — Mon, 16 Jul 2007 15:34:46 +0000

I like your general take on this but have a comment on “validation.”

I think you don’t need statistical validity with observation-based testing because the validation is really the expertise of the expert. In other words, you observe something that is, like you said, a “big” or “obvious” issue. That assessment validates the finding.

However, with A/B testing, there’s a big caveat: are you measuring preference or performance? In other words, are you banking on what people say they like or say they’d do? Or are you measuring what you observed them doing, and are comfortable that that’s what they’d do independent of any test effect?

I’ve always found that hard to achieve, and have generally avoided using 121 testing to find preference for one design/solution over another.