This chapter covers situations where the participant becomes agitated, annoyed, or disparaging toward you or your organization. These include situations such as a participant who reacts poorly to your neutrality, a participant who is frustrated by a prototype’s limited functionality, and a participant who seems to have an agenda against your organization. For each situation you’ll learn what to do, what to say, what not to do or say, and how you might be able to avoid it in the future.
usability agitated participant; usability annoyed participant; usability angry participant; usability prototype; user research participant
11.1 Remote participant is obviously distracted
11.2 Participant is distressed by a personal line of questioning
11.3 Participant insists that she would never do something
11.4 Participant is frustrated by the prototype’s limited functionality
11.5 Participant seems annoyed at your neutrality
11.6 Participant does not seem to respect you or take you seriously
11.7 Participant becomes insulting or has an agenda
11.8 Participant becomes agitated by a product’s usability issues
If the participant becomes agitated, annoyed, or disparaging toward you or your organization, it’s time for you to take the wheel of the session. These situations require you to diffuse the participant’s frustration, and then monitor the session going forward to ensure the rest of her experience is not negatively affected.
First, give the participant a subtle hint that you know something else is going on. Asking something like “Is this still a good time for you?” gives her an out in case she really does have something more important to deal with but is too polite to say so. It also gives the participant a chance to share any complications within her environment. For example, participants within corporate environments (if you’re talking to employees who are participating from work) often find it impossible to truly shut out the rest of the world for your research, and may be multitasking as you’re talking to them. The participant may tell you that she has to keep her email open during the session, or that it’s a very busy time and she may need to stop occasionally to talk with coworkers. In this case, you can offer to reschedule but you may need to accept that even so, she may still experience similar distractions.
If the participant is willing to continue and seems genuinely interested in providing feedback, but is dealing with distractions outside of her control, focus on the most important tasks/questions so you can make the most of her limited feedback.
If the participant doesn’t respond to your hint or you still get the sense that she is distracted, make a judgment call on whether or not to continue the session and, if you do continue, what you can adapt to fit these circumstances. If you think the session may be salvageable, consider asking the participant questions that are more directly targeted to what she does (e.g., how her process works today or what parts of the product she uses most often) instead of continuing with your preplanned tasks. Getting her to talk about herself may help increase her engagement in the session. If this works, you can then move back to your study plan, using the knowledge you’ve gained to tailor what comes next. However, if she still seems distracted, tell the participant that you’ve covered everything you’d planned, and then wrap it up early.
“<Participant>, is now still a good time for you?”
“It sounds like there’s something else going on. Do you need to take a quick break before we continue with <task/question>?”
“There seem to be a number of things competing for your attention right now. Are you able to continue with this study? If not, that’s fine, and we can reschedule.”
“I know you’re very busy, so I appreciate you taking the time to give us feedback.”
Be upfront about your expectations for the session so the participant can reschedule any competing priorities and plan accordingly. But, recognize that even this may not be enough and that interruptions could still come up.
During your recruiting process, ask the participant for the “best” time to conduct the session, and schedule her session for that time. Check with her the day before to ensure that the scheduled time still works.
Think about the participant’s expectations. If the participant was expecting to only talk in general about financial websites and is distressed by your questions about estate planning, try to make your questions broader and less personal to be more in line with her expectations. If this isn’t possible, it’s best to just drop the topic even if it means losing that data point. If the participant signed up to participate in an interview about online dating, then you can soften the personal nature of the questions you’re asking, but she shouldn’t be too surprised about the topic you’re exploring.
Let the participant know that if there are any questions that she’s not comfortable answering, she should tell you.
Reassure the participant about how her answers will be used. For example, tell her that her feedback will be aggregated with the answers of other participants, and her name will not be associated with it. This reassurance will help make the participant feel as comfortable as possible sharing what she is willing to disclose.
If the participant seems afraid of getting in trouble because of her answers (e.g., because she is an internal employee and the research is for her organization), add to your reassurance a reminder that she’s not being evaluated in any way on how she approaches her work or any other aspect of her life.
If the participant is especially uncomfortable or agitated by your questions, offer to talk to her about it in private, with any recording equipment turned off. If she seems really rattled, take a break, and when the session resumes, soften the personal questions or move on to another area of your study plan. If the participant still seems upset or agitated, bring the session to an early end.
“Let me know if you’re uncomfortable answering any of my questions. While we want to learn as much as possible from you, I also want to respect your privacy and your comfort.”
“I’m sorry, I didn’t mean to pry with that question. Let me rephrase.…”
“Please know that anything you say will be kept confidential.”
“Thank you for your feedback so far. I know these topics may be difficult to talk about, but your feedback will help us understand how to make the product better for you and others like you.”
While this is similar to a participant who can’t or won’t do something that was a key condition of the recruit (see section 4.2), this participant does match your recruiting criteria.
Take advantage of this opportunity to learn more about why this falls outside of what the participant does or would expect to do. You might learn something that will be incredibly valuable for your team about their users, especially if this task is a key component of the product. Ask her to clarify (and demonstrate if possible) her current involvement in the task or process. What are the steps that she takes, and the tools she uses? If the person who normally does the task isn’t there, what would she do?
In a contextual inquiry, if you need feedback specifically on this task or process, ask the participant if she can point you toward the person who actually does the task that you’re interested in, and see if you can schedule time with that person. You may be able to adjust your study plan to get some other relevant information from the participant—especially since site visits can be tricky to coordinate—but even if you can’t, be sure to provide her with the promised compensation!
Explain that while you understand that this task may not be something she does today, you’d still like her to attempt it so you can get feedback on the interface from the perspective of a new user.
If the participant pushes back (e.g., because she is more senior or more junior than the person who typically does this task), let her know that you understand, but her feedback would still be useful in making the product better by helping you understand what type of issues a new user may experience.
If she continues to push back or becomes obstinate about continuing, go ahead and skip to the next task. If time permits after you finish the rest of your study plan, you can ask the participant if she’d be willing to return to this task/process so you can get some feedback.
“Are you involved at all in <task/process>? If so, can you talk me through what you do, and what tools you use? Feel free to pull up any tools that you use as part of the process.”
“I understand that you don’t typically do this. However, for the sake of today’s study, I’d like you to play the role of someone who does <task/process> so we can understand what the experience would be like for a new user.”
Don’t move on to another task right away. Take advantage of the moment to understand more about the participant’s role, and how the task/process gets performed (and by whom).
Don’t push too hard if the participant is unwilling to play the role of a new user. Some participants have a harder time than others playing a role, and there may be political situations affecting her willingness to attempt this.
During your recruit, ensure that your criteria are detailed enough about the responsibilities and roles participants must have, especially for your key tasks (see section 15.1 for more on this). Also, reinforce these criteria in your communications with scheduled participants so they know what kind of responsibilities you’re expecting them to have. It’s better to discover a mismatch between user types and tasks before you begin a research study!
Refer back to what you told the participant in your briefing about her interacting with a prototype that has limited functionality instead of a finished product. Acknowledge that these limitations may be frustrating, especially if there are sections that she’s really interested in seeing or being able to provide feedback for. Gather her expectations for those missing areas; even if it’s outside the scope of your research, it will help the participant feel appreciated and may help reduce her frustration.
If the participant seems concerned that the missing or broken functionality indicates quality concerns with the product, reassure her that the prototype is not the same as the finished product. The prototype is just showing a limited set of features so you can get feedback on those specific areas.
Let the participant know that it’s helpful for you to see where she is trying to go, even though the functionality isn’t available. Emphasize that knowing her expectations for how an area of the prototype works is useful and ask for her to go into detail about how she would want those specific areas to behave.
Be patient with the participant, even if she has complained about the lack of functionality multiple times. Use an appreciative, sincere tone.
“I’m sorry that we couldn’t get everything built out in this prototype for today. But your expectations for how it should work are important too. Knowing where you expect things to be and what you’d expect to see if those areas were working is very useful for us.”
“As you can see, not too much is working in the prototype today. But thank you for being patient and just keep showing me where you’d go for these tasks and describing your expectations.”
Make your prototype as functional as possible given your available time and resources. For more discussion about how to build successful prototypes, see section 15.3.
Set clear expectations with the participant at the beginning of the session about what she’ll be using. A participant who is expecting a fully functional product may be horrified by a medium fidelity prototype with what she perceives as “broken” functionality. Reinforce those expectations when a participant tries to do something that isn’t supported by the prototype by quickly acknowledging that the area isn’t built out right now, but you’d like to hear what she expects to see/happen. By being proactive and redirecting her energy, you can limit the time that a participant spends wondering if the prototype is broken.
This annoyance typically happens with first-time usability participants who may not understand the reasoning behind your moderating approach. Participants may feel annoyed, perhaps because they’re genuinely looking for answers or help and aren’t getting any. Or they may feel patronized or challenged by what they perceive as you avoiding their questions.
Acknowledge the participant’s frustration and the artificiality of the situation with a friendly and understanding tone.
The next time you’re asked a question:
Consider your words carefully before answering instead of immediately turning it around with “What do you think?” or “What were you expecting?”
With each question, subtly change the way you respond. For example, one time ask, “So—I’ll turn that around on you again. What are you thinking?” and another time ask, “Tell me what you’re expecting?” If you’re not confident in your ability to rephrase on-the-fly, write down some responses ahead of time in your study plan that are similar in meaning but use different words.
Use paralanguage such as intonation to show that you’re genuinely interested in an answer. If you’re comfortable, you can use a bit of humor in your response. For example, “I know I sound like a broken record here! I really do want to understand your expectations though.”
If the participant is really annoyed, do one or both of the following things:
Reassure the participant that it’s really valuable for you to hear her questions. Remind her that if she was attempting these tasks on her own you wouldn’t be sitting next to her, so you’d like to see what she would do if you weren’t there.
Take responsibility by apologizing once for your constant neutral responses. Avoid apologizing more than once though, as that may create awkwardness and potentially bias the participant’s feedback.
“The feedback you’re giving is really valuable for our team. We need to know what questions people will have when they try to use this, so the product can do a better job of providing those answers.”
“I know it’s odd to have me keep turning the question around on you, but I really want to play a neutral role here and just understand your expectations. Thanks for being patient with me.”
“If you were doing this on your own and I wasn’t sitting next to you, what would you do?”
At the beginning of the session, set the expectation that you may not be able to answer questions during the session and that you may turn any questions back on the participant to gauge her expectations. Stress that even if you do so, you’re genuinely interested in hearing her ongoing feedback and any questions she’d have while using the product.
Be unfailingly polite. You can think of it as a challenge to get the participant to leave the session with a newfound respect (or at least a grudging acceptance) for you!
If the participant pointedly asks you how old you are, or for any other personal information about yourself, try to avoid directly answering the question. For example, if she asks how old you are, you can say that you’d rather not answer and quickly redirect to another topic. Providing personal information shifts the balance of the session away from your control and may take the participant further off topic. However, if you do not answer the question it may be perceived as rude, so you should provide a quick answer followed immediately by a redirect back to the task or question at hand. If you’re comfortable, you can also try using light humor. For example, if the participant asks what year you were born in, you can say, “I’m a baby boomer but that’s neither here nor there! Can you tell me more about.…”
Consider playing up the role that the participant seems to have cast you in by asking the participant to help explain things to you. For example, if the participant seems to think that you’re too young or inexperienced to understand the kind of issues she’s facing with retirement, respond immediately by asking her to tell you more about her situation and the challenges she’s facing. This approach has the added advantage of flattering her ego, and may help her “buy in” to the session.
“I’d rather not answer that question right now, but I would like you to tell me more about.…”
“You alluded to some of the factors that you’d consider important when making this decision. Tell me more about those factors and how they would influence the choice you’d make.”
“It’s really important for us to get the perspective from someone with your experience and expertise, so we’re really grateful you found the time to come in. We want to make sure that this product will be designed to meet your needs.”
Establish your credentials and your role at the beginning of the session. Let the participant know that you’re a neutral observer, not a subject-matter expert.
Treat the participant with respect by using her title (e.g., Dr. Smith) until she tells you otherwise.
If the participant has a high-level role or a lot of authority in her job, acknowledge that the role she plays in her organization is a critical one so you’ll make good use of her time.
If the participant seems to be venting about your stakeholders or your organization, provide her with a focused outlet for her venting. For example, you can acknowledge that she seems to be upset about bad experiences she’s had, and suggest taking the next few minutes to have her share what happened and then proceed with the rest of the session.
If the participant insults you directly (e.g., “You’re really bad at this”), take a moment before responding. You may be in shock but need to carefully think through your next step so you avoid escalating the situation. Try to either ignore the comment and move on without acknowledging it, or place that topic out of scope (e.g., “That’s not relevant to our session today”) before quickly moving on to another area of discussion.
Try your best to not react, and maintain a neutral expression and tone. This may be difficult to do, especially if you feel personally attacked, but do your best. Take a deep breath, control your face, and imagine the participant’s comments rolling off of you like water on a duck’s back—don’t let them stick.
Take a short break if necessary so you can recover and maintain neutrality in front of the participant. Use the break to consider your options for what you can do to improve the session once you resume.
If the participant’s insult was in the context of what you’re doing for the study (e.g., a derogatory remark when you ask her to use a website she strongly dislikes), clarify your role and responsibilities as a neutral observer and what your expectations are for what she’ll be doing. Apologize for any misunderstandings and ask if she has any more questions about how the session will work.
If the participant’s comment seemed sparked by what you were asking her to do or talk about (e.g., “Only an idiot would even try to do that”), change the topic of conversation or redirect the participant to the next task or section of your study plan.
If the participant continues to be insulting or disparaging toward you or becomes confrontational, end the session early and pay her any promised compensation. Remember that you need to look out for your emotional well-being and that additional data point you’d gain by sticking with the session is not worth it.
For a venting participant:
“You keep alluding to some really bad experiences with <organization>. Why don’t we take the next few minutes for you to share your experiences? However, after about five minutes, we need to get back to my list of questions. Does that sound okay?”
“Thank you for sharing that experience. Part of why this research is being performed is to help improve what users go through, so your feedback will hopefully keep others from going through bad experiences. So, let me ask you <return to your study plan>.”
For a participant who deliberately insults you:
“This topic is not relevant to our session today. Let’s go back to our questions/tasks, unless you’d like to take a short break first?”
To redirect a participant:
Don’t get pulled into the participant’s agenda.
Don’t insult or get into an argument with the participant, even if you really want to! Remember to stay professional.
If the participant insulted a group of people who you strongly identify with, don’t bring up yourself as a counterpoint to whatever the participant said.
Maintain a calm, neutral demeanor, even if you’re startled or nervous about the participant’s agitation. Keep the tone of your voice neutral and calm, and avoid any sudden shifts in body language. Avoid anything that might increase the level of agitation.
Acknowledge the participant’s frustration, and thank her for sharing her feedback. Slowly adjust your position to angle toward her and make eye contact. Reassure her that even though you know she’s irritated, it’s helpful to hear about these issues so the product can be improved.
Redirect her away from whatever is causing the extreme agitation. For example, if she’s frustrated because she can’t figure out how to accomplish a task, follow the recommendations in section 6.4 on giving the participant an assist to help her continue, or just move on to the next task/question that covers a different area of the product.
If the participant continues to be agitated, take a short break so she has a chance to calm down. When you resume the session, move on to a task/question that covers a different area of the product.
“It’s really helpful for us to hear what areas are confusing or frustrating so they can be improved in future versions. So, we really appreciate you going through all this with us!”
“Thank you for your feedback so far. Let’s take a quick five-minute break so I can see if any of our observers have any questions for you.”
At the beginning of the session, tell the participant that if she finds anything confusing or frustrating, you want to hear about it so the product can be improved so others don’t experience the same issues. Making sure the participant feels that she has permission to share her frustrations may reduce any agitation she feels later in the session.
52.14.239.155