In the Trenches
Six Steps for Handling Situations
Abstract
Once a session begins, you can follow six simple steps to figure out the best course of action when something unexpected, tricky, or sticky happens. These steps include taking a moment to evaluate the situation, resolving any threats to physical safety (yours or the participant’s), verifying that you’re not causing or magnifying the situation, checking the participant’s comfort level, using careful language and tone to probe on the situation and begin to resolve it, and regaining control to bring the session back on track.
Keywords
user research safety; user research participant comfort; user research ethical issues; user research pretexts; user research moderator language; user research moderator tone; user research what not to say
2.1 Take a moment to evaluate the situation before jumping to action
2.2 Resolve any threats to physical safety
2.3 Verify that you’re not causing or magnifying the situation
2.4 Check the participant’s comfort level
2.5 Use careful language and tone to probe on the situation and begin to resolve it
Once you’ve understood your role as the moderator and the different styles of moderating, you can start thinking about how to handle the different kinds of situations that may arise during the session. Some of these situations will challenge your interpersonal skills; others will push the limits of your troubleshooting and crisis-handling capabilities. Although there are common circumstances you’re almost guaranteed to encounter, we can’t predict what else might happen. Even after our years in user research, we’re still surprised (and sometimes delighted, shocked, or horrified) by what happens when we’re one-on-one with a participant. This chapter provides a set of guidelines (as shown in Table 2.1) to help you decide exactly what to do when something unexpected, sticky, or tricky happens during your session.
Figure 2.1 Examples of moderating behaviors. The Obliviously abrasive (left) and the Buddy-buddy (right). © 2013 Mark Ainscow, used with permission.
2.1 Take a moment to evaluate the situation before jumping to action
You’re running an interview with a remote participant over the phone. The session seems to be going well until the participant doesn’t respond as quickly to your questions. You hear him whisper to someone in the background but can’t understand what he says. What, if anything, should you do?
When you encounter anything outside of expected session circumstances, take a moment to step back and evaluate the situation. As the moderator, you keep track of so many things that it can be difficult to take that time. But doing so is vitally important. Why?
You need to make sure that you understand the cause of the situation so you can address it appropriately. Rushing to do or say something may lead you to inadvertently make the situation worse or further compromise the results of the session.
The situation may resolve itself. In this remote session example, a few moments may be enough for the participant’s whispering to stop and for you to be able to move on with your questions. Also, consider the example of a usability study participant who becomes confused or frustrated. A delay of a few extra seconds is often enough time for a participant to find his bearings and continue without any intervention from you. This delay may also be all a distracted participant needs to reengage with a task or question on his own.
If the situation persists, those additional seconds will keep the participant from seeing you sweat (or, in a remote session, hearing you fumble your words as you try to figure out what to do). Remember that, as the moderator, you establish the pace for the session. Take this moment to think through your options and prepare what you’re going to do and say if the situation continues.
This moment may feel like a lifetime to you, but it doesn’t necessarily feel that way to the participant! Even in a potentially life-threatening situation (e.g., a fire alarm goes off), take a moment to gather as much information as possible before proceeding. Your need to respond appropriately should outweigh your instinct to react immediately. Remember that you’re responsible for the participant’s well-being as well as your own.
If the situation is especially tense or you’re not quite sure what’s going on, you may want to take your moment in another room, away from the participant. Taking a break is an incredibly powerful tool that we’ll discuss more in section 3.7. A break gives you a chance to think about what’s happening in the session without the pressure of monitoring everything else. If you have observers, you can use this break to check with them and get a “gut-check” on what you think is happening and why. The participant may also benefit from a few minutes to cool off, or deal with his distractions.
Table 2.2 shows examples of what you can say if you need to take a break. Note that you can use these phrases even if they aren’t entirely true—consider them mild deceptions for the greater good. Pretexts are discussed further in the sidebar “Using a Pretext.”
USING A PRETEXT
While you should avoid misleading the participant, there are times when the truth is going to be harmful. For example, if you notice that the participant does not have the characteristics you need and you want to consult with your stakeholders (who are in another room) about what to do, you should not express your concern about his qualifications directly to the participant. Being honest may hurt his feelings or make him nervous about participating. Instead, make an excuse to leave the room.
Technology can be blamed for many things and provides a convenient excuse if you need to leave the room or end the session early. Just make sure that you have something realistic to blame so that your reason for leaving isn’t obvious. For example, if you say, “The computer isn’t working correctly,” the participant may realize that you’re lying because he can see that it actually is working. But if you say, “Our prototype isn’t working the way it’s supposed to,” he doesn’t have any reason to know that it is just an excuse.
Examples of what you might say include:
“It seems like we may be having technical issues with our recording equipment. Let’s take a short five-minute break while I run next door to look into it.”
“I apologize, but I just noticed that there is something wrong with our prototype. Please excuse me for a few minutes while I go to the other room and see if anyone from our team knows what’s going on.”
“I’m sorry, but it looks like the product I wanted to get your feedback on isn’t working. I have a few more questions for you, but after that, we’ll end the session early. You’ll still get your full compensation though!”
If you don’t feel comfortable blaming the technology or it isn’t applicable (e.g., in an interview where you aren’t using any equipment), you can use the study itself as a pretext:
“I meant to give you your compensation at the start of the session. Let me actually go grab that now. Let’s take a break for a few minutes.”
“I just noticed that I’m missing a page of my questions. Let’s take a quick break so I can run next door and print that out.”
“The rest of these tasks/questions aren’t applicable to you/your role, so we’re going to skip them. This means we may be done a little bit earlier than expected, but don’t worry, you’ll still receive your full compensation!”
Table 2.2
For a usability study with a multiroom setup:
“We’re at a good stopping point right now. Let’s take a short break and resume the session in five minutes.” (If appropriate given your setup, you can add: “Feel free to leave the room to get a drink or go to the restroom while I’m gone.”)
For an onsite and remote sessions:
2.2 Resolve any threats to physical safety
In the middle of a contextual inquiry, you hear a low rumbling sound. You take a moment to evaluate the situation, and quickly realize that you’re experiencing an earthquake. What do you do next?
When a situation involves a physical threat, you need to respond to and address the threat as soon as you’ve taken a couple of seconds to understand what’s going on. Respond quickly and appropriately, even if that means that you’re unable to continue the session. The safety of you and the participant takes priority over your data collection needs!
Keep in mind that the participant may not know what to do, especially if he is in your facility. You must take responsibility for his safety as well as your own. For example, if a fire alarm goes off or your building is evacuated, bring the participant with you as you follow standard evacuation procedures. While the participant may end up seeing your observers, your top priority is to keep him from physical harm instead of maintaining the integrity of the session.
In the example of an earthquake, you should instruct the participant about where to go (e.g., under a desk or in a doorway). If you’re moderating a usability study from an adjacent control room, and you think it’s safe to spend another couple of seconds using the microphone, tell the participant that you’re experiencing an earthquake and instruct him to go under the desk or in the doorway immediately.
If you’re in the participant’s space, follow his lead to exit the building safely or follow the appropriate safety protocol for the event that you’re experiencing.
Depending on the severity of the situation and safety of the environment, you may want to cancel or end the session rather than asking the participant to wait around until the situation is resolved. If you do this, be sure to give him his full compensation and thank him for the feedback provided. If the environment is safe, and the participant is not visibly shaken and willing to continue, offer to take a short break before resuming so you and the participant can both regain focus.
2.3 Verify that you’re not causing or magnifying the situation
During an interview, the participant starts making snide remarks after each question you ask. You’re not quite sure why he is responding that way. The issue doesn’t resolve itself, and taking a break doesn’t help. There is no physical danger to resolve but you’re worried about the participant’s attitude and its effect on the feedback he is providing. What, if anything, should you do?
Let’s step into some awkward territory for a minute, but we promise, it’s for your own good! Sometimes a situation is magnified, if not created, by something that you’re doing. Part of being a good moderator is being humble enough to constantly evaluate and reflect on how you’re interacting with participants. Before you jump to a conclusion about why the participant is behaving in a certain way, try to detect anything you may have done that led to this behavior.
Table 2.3 shows some examples of problematic moderator behavior that we’ve observed, the moderating styles (as identified by Table 1.1) that may encourage this behavior, and the effect it can have on the participant’s comfort level.
As discussed in Chapter 1, there is a delicate balance between the goals and method of your study, the type of participant you have, his current emotional state, and the appropriate moderating style to use. These last two elements in particular can change multiple times throughout the session. You should always carefully monitor how the participant responds to avoid making him uncomfortable. Again, part of the art of moderating is in evaluating your technique throughout the session and adapting it as necessary.
To find out if you’re creating or contributing to a problem:
Are you too close to or too far from the participant?
Are you maintaining eye contact, or are you making too much eye contact?
Have you been yawning or glancing at your watch?
Are you sitting upright, or are you slouched down in your chair?
Have you been frowning or grimacing instead of maintaining a friendly, neutral expression?
2. Replay what you’ve said during the session:
Have you neglected to reassure the participant that you’re listening to him? Have you been chiming in with the occasional “mm-hm” to encourage him to continue talking?
Have you been talking too much, and not letting the participant talk?
If running a usability study with a think-aloud protocol, did you give the participant enough time to respond before reminding him to think aloud? (See the sidebar “Learning to Enjoy the Silence” for more on this.)
Did you stop using the participant’s language and start using language from the product’s user interface, domain, or any other jargon he may not understand?
Are you using close-ended or leading questions that “give away” the answer you want or expect to hear?
Have you been laughing with or making jokes with the participant?
3. Replay your paralanguage—how you said what you said:
Did you sound judgmental about an opinion or detail that the participant shared?
Did you insinuate that the participant did something incorrectly?
Could your tone be heard as patronizing or disingenuous?
4. Evaluate your note-taking behavior:
Are you shuffling papers while the participant attempts tasks or answers your questions?
Have you been taking notes only when the participant says or does something “wrong” or unexpected?
5. Check the physical setup of the room:
Is the temperature too hot or too cold?
Is there anything in your equipment setup that the participant seems to be struggling with (e.g., a monitor that is pushed too far back on the table, or a chair that is set too high or too low for the participant)?
We’ll talk more about how to build awareness of your own behaviors while moderating in Chapter 16.
2.4 Check the participant’s comfort level
Once you’ve verified that the situation you’re seeing hasn’t been caused or exacerbated by your own behavior, check to see if the participant’s degree of comfort has declined to an unacceptable level.
Most user research starts out being somewhat uncomfortable for the participant. He may arrive without knowing exactly what he’ll be asked to do, or he may worry that he is being tested in some way. If you are coming in to his space, he’ll be unable to get away from the typical hassles of his environment, which may become magnified by the uncertainty of what he’ll be asked to do during the session.
Unless you’re doing very specific research that deliberately puts users in uncomfortable situations, you want to minimize, if not altogether eliminate, anything that compromises the comfort or safety of the participant. Part of your job as a moderator is helping the participant feel comfortable participating and providing his feedback, and ensuring that he remains comfortable—physically and psychologically—during the session.
PROTECTING PARTICIPANT RIGHTS
Most professional organizations whose members interact with nonmembers in a professional capacity, including the User Experience Professionals Association (UXPA), have a code of conduct or set of ethical guidelines to govern their members’ behavior. These guidelines are in place to ensure the ethical behavior of the professional (i.e., you as the moderator), and protect the rights of the nonmember (participant).
As a user researcher, you need to keep your ethical obligations to the participant front and center throughout the session. Remember, when the participant comes to your session, he may have little to no information about what he’ll be doing and why he’ll be doing it. It’s your job to provide him with this information before he begins, and obtain his consent to participate.
This informed consent lets the participant know what he’ll be doing as part of the study, what he’ll receive for participating (compensation), and his rights (e.g., to take a break or leave at any time). Even though you should have him sign a consent form before the session begins, review the key points verbally as part of your briefing at the beginning of a session.
We recommend reviewing the UXPA’s code of conduct at http://www.usabilityprofessionals.org/about_upa/leadership/code_of_conduct.html, which outlines how to behave ethically as a user researcher. As a quick cheat sheet, here are the key things to remember when interacting with your participants:
Do no harm, and if possible, provide benefits. As discussed in Chapter 1, there is always some level of stress associated with participating in user research. However, your responsibility as a moderator is to keep that stress—whether physical, mental, or emotional—from reaching an unreasonable level.
Act with integrity. Be respectful toward the participant, no matter who he is or what he does.
Respect the participant’s privacy, confidentiality, and anonymity. Make sure he provides his informed consent, and that he knows the steps you’ll take to avoid disclosing his personal information.
Ensuring the participant’s physical comfort is usually fairly easy to accomplish. Let him know that he is welcome to take a break at any time and, if possible, offer him a beverage. If he seems uncomfortable in the research space (e.g., because the chairs are at an awkward height or size for him or the air conditioning is running high), make whatever adjustments you can to keep the environment as welcoming and stress-free as possible.
The participant’s psychological comfort can be more challenging to maintain, especially when he is asked to interact with a product that has major usability issues or he is being interviewed about a sensitive or private situation. However, keep in mind that if a participant is struggling or growing frustrated with a usability study task, his comfort is not necessarily being threatened. In fact, that struggle often fulfills a common goal of user research, which is to identify problem areas. The participant’s struggle serves the purpose of making the product better for future users.
If, however, the participant berates himself for being stupid or unable to figure something out, or exhibits more than one of the signs of an uncomfortable participant (Table 2.4), you need to alleviate his discomfort. Remember that a participant who leaves a session feeling disgruntled or unhappy is an unhappy customer (or potential customer). Given how easy it is to share negative reviews and experiences via social media, you want to avoid any potential damage to the credibility of your research or your organization.
Table 2.4
Signs a Participant May Be Uncomfortable.
He increases his physical distance from what’s being evaluated (e.g., chair pushed away from the table or keeping a physical product at arm’s length).
He blames himself or apologizes for being “stupid” or inexperienced.
He has difficulty speaking clearly.
He exhibits signs of emotional distress, such as tremors, crying, or excessive sweating.
He constantly glances around the room or at the cameras.
He asks to end the session early.
He pays excessive attention to distractions (e.g., his email is open in the background and he keeps checking it).
If the participant seems uncomfortable, be careful not to assign any blame to him. Remember that he is likely in an unfamiliar environment (to him), or may be stressed to show a stranger (you) around his comfortable environment. He may be under a lot of stress unrelated to the session, such as childcare issues or tight work deadlines. Even if there is nothing external
Watch Video 1 to see an example of a moderator interacting with a self-blaming participant during a usability study. The participant is very nervous about the session and is looking for affirmation.
Visit our website (http://www.modsurvivalguide.org/videos) or use your QR reader to scan this code. 
causing his discomfort, the research session itself can create some stress no matter what you do to avoid it.
To help the participant feel more comfortable,
Reassure him that he is not being tested, and that anything that frustrates him is valuable information for the product team. Remind him that his feedback will make the product easier to use for future users. For a contextual inquiry or interview, you can reassure him that his feedback is confidential and will ultimately be used to make his own life/work better.
If you’re running a usability study, avoid using the term test when referring to your research. A test is something that can be failed, and may place additional pressure on the participant. He may not understand that you’re testing the product and not him. Instead, use study, session, or evaluation of <product>. Likewise, you may consider calling each usability task a scenario instead of task.
If the participant crosses his arms or moves away from you (or the product being evaluated), ask him to tell you what he is thinking. This question gives him permission to express the reasons for his discomfort, and may reveal if he is uncomfortable because of something you asked him to do.
Ask the participant if he would like to take a short break, or go ahead and suggest a break.
If the participant reaches a point where he asks to leave or end the session early, you’re obligated to wrap things up. Be sure to give him his agreed-upon compensation and thank him for his time and help. You can check on him afterwards (using whatever means you originally scheduled him for the session; if he was scheduled by a recruiter, ask the recruiter to follow up). If appropriate (e.g., if he was distraught due to external circumstances), you can even offer to reschedule for a more convenient time.
2.5 Use careful language and tone to probe on the situation and begin to resolve it
Once you’ve established that you’re not causing or magnifying an issue, you’ll probably need to talk with the participant to try to understand more about, and hopefully resolve, the situation. One of the most important principles in moderating is to let the participant do most of the talking, but when you do speak—whether it’s to provide direction, probe for more detail, or give neutral reassurance—you need to choose your words, and your tone, carefully.
To choose the appropriate language, we suggest reviewing and memorizing some prefabricated responses that represent moderating best practices. Prefabricated responses are valuable for interacting with participants while maintaining neutrality and consistency. By memorizing at least some of these phrases and reviewing a list before your study, you save the cognitive effort of improvising appropriate responses on the spot. As we start reviewing specific situations in Part 2, you’ll be amazed at how flexible these short phrases can be.
The phrases we use most often include:
“What are you trying to do right now?”
“What are you thinking right now?”
“How did this compare to your expectations?”
“What are your thoughts on how that worked?”
“That link isn’t working for today’s session. What would you expect to happen if you clicked it and it was working?”
“Thank you for that feedback, but for the sake of time, please move on to …”
Since you need to be careful about providing positive feedback to participants, we’ve found the following phrases to be relatively neutral. When used judiciously, the phrases are capable of acknowledging to the participant that you’ve heard his feedback and are giving him a small amount of reassurance that the feedback was helpful:
“Okay, thank you,” or “Okay, thanks for that.”
“That is good to know,” or “That is helpful for us to know.”
Careful use of these phrases will address the majority of typical issues you see during your research. Just as important as what to say is what not to say, which we give examples of in the sidebar “What Not to Say.” Of course, you’ll encounter situations where these neutral phrases may not cut it, or if a participant doesn’t act as expected when given one of those phrases. In these cases, keep in mind these three guidelines:
1. Be as unbiased and nonjudgmental in your response as possible. No matter how inflammatory or provoking the situation is, or how much you agree (or disagree) with what the participant is saying, it is your responsibility to remain neutral. Remember that it’s not just what you say, but also how you say it, both in intonation and body language. For example, if the participant is describing a personal or cultural belief that you find repugnant or inappropriate in response to your interview question, try to keep your face still and your voice at an even neutral tone as you ask your next question.
2. Build on what the participant says rather than negating it. As mentioned in Chapter 1, one of the fundamental principles of improv is to never disagree with something that has been said, but instead to say “Yes, and.…” This principle means that you’re always building on what someone else is doing rather than negating it. In a research session, your “Yes, and …” is less about agreeing with what the participant has said and more about acknowledging that he is been heard. For example, if the participant asks if he has performed a task correctly, respond with a prefabricated phrase instead of ignoring the questions. For example, “So I’ll actually turn that question around on you: Do you feel that you’ve found the information that you’re looking for?”
3. Be as graceful as possible when redirecting or reacting. When the participant wanders off track or no longer provides useful feedback, your paralanguage will set the stage for his reaction. You don’t want to imply that he has done something wrong. Here’s one of our favorite examples of a moderator–participant exchange that we’ve seen, showing how your word choices and intonation can have a negative effect:
Moderator: How would you do <task>?
The rising intonation implies a questioning of the participant, and may cause him to feel judged or that he should rethink his answer. Instead, a graceful reaction would be to say “okay” with a neutral tone and move on, or ask him if there are other things he would try if the first attempt didn’t work. If he is interacting with a prototype, you could also let him attempt the task, ask how it compares to what he expected, and then ask if there are any other ways he might try to do the same thing.
2.6 Regain control to bring the session back on track
Hopefully, what you’ve done so far has jumpstarted the session. However, if the participant feels out of control or continues in a direction that is not useful to your research goal, it’s your job to steer him back. As the moderator, you’re always responsible for maintaining control of the session. This means accepting responsibility for changing the tone or providing whatever support is necessary to bring the session back on track.
Remember that you’re not the participant’s buddy. You didn’t set up this time with him for a gripe or therapy session. It’s your job to keep him from treating it that way. At the same time, some of the art of moderating includes redirecting a session and treating the participant’s feedback with grace and respect.
So how do you reestablish control of the session? For most one-on-one methods, a specific restatement or refocusing of a task or question is enough to get the participant back on track:
“Thank you for going into that level of detail. For the sake of time though, I’d like you to return to <attempting/answering task/question>….”
“Thank you for explaining that, it’s helpful for us to hear. We’re a bit pressed for time so I’d like to direct your attention to….”
You can also bring the participant to a different task or question if you think he may be getting stuck or hung up on something in particular.
If your first attempts to regain control don’t work, you may need to use stronger language to rein the participant back in:
“I apologize for interrupting, but it’s really important that I get your feedback on this <task/topic>. If you don’t mind, let’s return to that, and if there is time at the end of the session, we can return to this discussion in more detail.”
Also, check that your body language projects an appropriate authority level. Maintain eye contact with the participant and angle your body toward him—this will help draw his attention and increase the odds that he’ll pay attention to you. However, never touch the participant, even as a way to get his attention or give him comfort. Remember that many people do not like being touched by strangers and the inherent power imbalance within a moderating session can make any touch completely inappropriate.
If the session is veering out of control through no fault of the participant (e.g., technical problems), avoid sharing your frustration with him and do your best to keep that emotion out of your voice. The participant may feel like you’re directing that irritation at him.
Another reason you may need to regain control is when the participant does not meet the desired criteria and you must adapt accordingly. One approach to this situation is to transition the session into a different type of research. This works especially well if the participant is an important customer, or part of a user group, who you want to feel validated and listened to. For example, a usability study with a customer who doesn’t match your target user profile may be transitioned into an interview that will provide more information about his profile. Be sure to discuss these options with your stakeholders so they know what’s happening and can provide input on other types of feedback they might be interested in receiving. If, however, the participant will be unable to provide any kind of useful input (e.g., if his role is entirely unrelated to your target), you may need to politely end the session.
No matter what situations you encounter, the steps discussed in this section should help you figure out the best approach to address and resolve them. When in doubt, think back to what you would want done if your positions were reversed—how would you want your moderator to handle the situation? Chapter 3 details some of the specific interaction patterns that you can apply to help with this resolution.