despiertos Ojos que no ven, ostia que te pegas…

User Testing

Smoke & Ashes - Tracy Chapman (The collection)

First thing, for those of you know who may not know me, I work as a usability consultant. And for those who haven’t even heard about usability, it’s kind of a science that tries to make people lives easier by simplifying machines, software, webs and many other stuff that you probably use in your everyday life.

What I basically do is user testing. I show a new or improved product to a random person, who could be a target of that product, and I watch how they use it, which reactions they have, how they feel… I suppose you get the point. I basically observe.

As this is my first post about my professional job I’d like to talk you about the human part of my job: The users. When preparing a test, we write down some questions, call some random people and ask them those questions. If they fit the profile we are looking for, we ask them to join a test and, obviously, reward them with different amounts of money that varies depending on some factors such as length of the tests, its complexity, etc.

Most of the times everything goes perfectly, but sometimes, things get pretty ugly. You must notice that leading a test is quite an uncomfortable thing. You must play the “alpha dog” role with someone you don’t know at all. And although people are usually nice and collaborative, I’d like to focus on difficult users.

The most common problem we face is a user that doesn’t show up, even having confirmed both date & time twice. Sometimes they call you and apologize for not coming, but in most cases they don’t even bother.

Then we have people that deliberately lie when we ask them the mentioned questions. Note that the interviews are by phone, so it’s pretty much difficult to notice if someone is lying. Just to put a drastic example: I’ve seen users coming to a web test that can’t even identify a single link. There was that guy that confused the Firefox’s toolbar button with a link. Yeah, as weird as it sounds. And those people where questioned if they surfed the net at least 3 days a week.

But if you ask me for the worst case I’ve seen, this is the user that showed up absolutely drunk. I promise you, the room smelled wine for hours. He was so drunk he couldn’t even handle the mouse.
The question is: what do you do in those cases? The amount of money you loose with one single of these bad users is considerable:

• Usually there are two people, besides the user, working in a test, the person who is with the user and another one observing the session remotely. Those people must, of course, get paid for their time, because it’s not their fault that no data can be obtained within that hour.

• There is the user’s reward: Even they have deliberately lied, you are morally (and legally, I guess) forced to pay them for that time.

• You must spend time contacting another user to person to do the test.

• You’ll have to pay this new user, the people who are working at the test, and so on…

Had you ever considered these factors? Do you have any particular experience you’d like to share?

Anything you’d like to ask? Go ahead!