Four Words to Improve User Research

Oct 1, 2003

Like any good user experience practitioner, I often spend time face-to-face with users of clients' products, services, and websites.

Unlike many practitioners, I use a research method that is a bit different from the established teaching of usability and HCI.

I'll admit upfront that I'm biased towards this method (note: always be wary of gurus whose ego is bigger than the building you're sitting in). But I'm biased because I've seen the results this method creates, both in the data that's generated and in the *business results* the client enjoys afterwards.

The method is the "listening lab": a more open-ended version of the traditional usability test. Listening labs generate *strategic* findings - not just tactics - and point the way to measurable *business* results - not just usability results like task success or time-on-task.

In their setup, listening labs are familiar: one-on-one sessions with a moderator and user, in front of a PC, typically in a lab facility with a one-way mirror, with observers sitting in the other room.

Listening labs differ from usability tests in one key way. This is the one simple thing that, if done properly, can transform the research and its results. Just four words.

FOUR WORDS TO IMPROVE USER RESEARCH:

Don't

define

tasks

beforehand.

When you walk into the testing facility as the moderator, or even as the client, *don't* have a script or list of tasks written out already. Maybe you were told to do so in a usability book, or by a "guru," or learned it at school. But don't do it.

Instead, when the test starts, interview the customer on how he or she relates to the website or product in question. (When, and why, do they typically use this service? What about competitors? What's the process they go through? What's a specific example from the recent past, or near future, that they will undertake with this service?) So, first understand their context: show that you're *listening*.

Then, with the customer's context in mind, give them a task to conduct - the same task they just described to you in the specific example. As they turn to the PC, ask them to keep talking, describe what they're thinking, and say why they're taking each action.

At this point, the customer hands you test data you never could get in a rigid, traditional usability test: tactical learnings, strategic thinking, and everything in between; almost none of which you could have *guessed* beforehand, had you written a script.

Because that's what a task list is, after all - the test moderator presuming to guess how customers use the service, without even talking to them first. It's the moderator setting a rigid framework, saying, "I know how and why customers are here."

But most websites are a strategic representation of the *business*. How can you presume to know how customers relate to the business, unless you ask them first? If your test moderator has a healthy case of ESP, or can bend spoons without touching them, by all means, define the tasks beforehand. Otherwise, try the listening lab method.

And remember that you *will* get tactical data. Listening labs *are* task-based - but only on the tasks most relevant to each user.

Strangely enough, most customers (within each customer segment) will VOLUNTARILY come up with the same tasks, and the same feedback, on the site - without any leading from the moderator.

I just finished two days of listening labs for a global Fortune 500 corporation. Creative Good is creating the customer experience strategy for the entire site, so in the listening labs we had respondents from all their major customer segments. By the fourth respondent in each segment, we were seeing the same tasks, the same feedback, the same results - created voluntarily by the customers themselves.

Here are some next steps:

- For you: In your next round of customer research, don't define tasks beforehand. Let the customers lead. (Another good four-word phrase, come to think of it.)

- For me: In my copious spare time, I'll try to write more about listening labs, since this is the topic I get asked about most (along with measuring business results) at the seminars, conferences, and companies I speak at.