Mix and Match
Your Moderation Patterns Toolbox
Abstract
When common, tricky, and sticky situations occur during a user research session, you can draw on a rich set of moderation interaction patterns to help address those situations. This chapter describes each interaction pattern, when that pattern might be appropriate to use, and how to apply it. Patterns discussed include taking responsibility, redirecting the participant, reassuring the participant, taking a break, and ending the session early.
Keywords
user research moderating patterns; user research assists; user research redirect participant; user research task question clarification; user research reassure; user research shift focus; user research take a break; user research end a session early
As we start describing situations that you may encounter (in Part 2), you’ll notice that we repeat some recommended actions over and over again. You can think of these actions as moderation patterns that can be applied to almost any situation, and modified, as needed. Table 3.1 lists all the patterns that we’ll discuss in this chapter. Some of these may seem obvious to you, especially if you already have some moderating experience under your belt. Even so, our goal is to help you think about when each pattern is appropriate to use based on the circumstances of the session and your desired outcome.
Most of these patterns aren’t mutually exclusive—you can use one or multiple at a time or within the same session. Given that you may encounter situations that build on each other (e.g., experiencing technical problems while the participant is already distracted by external circumstances), you may need to apply multiple patterns within a short time period.
3.1 Take responsibility
Taking responsibility means recognizing that, as the moderator, you’re in charge of the session, and then using that power wisely.
When to use
When something happens during a session that a participant may not know how to react to, or when a misunderstanding is encountered, you should step up and take responsibility. This pattern may be appropriate:
To keep the participant from feeling badly about something she did.
If the participant seems to have misunderstood something you asked her about, or to do.
If the participant is approaching emotional or physical discomfort or physical danger.
If there is an emergency where you have to assist the participant to ensure her safety.
What to do
This pattern means that you take responsibility for anything that, from the participant’s perspective, is unexpected during the session. For example, if you need to skip tasks or end a session due to something about the participant’s behavior (e.g., she brings a baby with her to the session or misunderstands a question), you can blame yourself for not having the setup to support her situation or for not choosing the best words. Putting the responsibility on you keeps the participant from feeling badly about herself.
This pattern also means that if something in the session seems to be going wrong, you should take, or absorb, responsibility for it. If the participant is complaining, don’t agree or disagree with her—just absorb and accept her feedback. If something else goes wrong, don’t blame your designers, developers, recruiters, or the participant—even if something is objectively their fault. As far as the participant is concerned, the buck stops with you.
You also need to take responsibility for the participant when it comes to her physical comfort and safety, especially in life-threatening situations such as fires or earthquakes. Especially if she is at your facility, it’s your job to watch over, keep track of, and direct her until she is safely on her way home.
Keep in mind that you may also need to use this pattern with your observers, especially if your observers are in the same room as you and the participant. If the observers are distracting or causing problems, you must take responsibility and resolve the situation to ensure the comfort of the participant.
See pattern in use
You can see this pattern being used in situations such as:
3.2 Clarify the task/question
By clarifying what you mean by a task/question, or asking the participant to reiterate her understanding of what you asked her to do, you’ll reduce misunderstandings and improve the odds of getting the kind of feedback you need.
When to use
Use this pattern when you want to make sure that the participant is clear about what you asked her. This pattern may be appropriate:
What to do
First, verify that the participant misunderstood the task/question by asking her to tell you, in her own words, what she is doing or describing. For example, if you asked a participant during an interview to tell you about the last time she purchased a computer and she starts talking about a recent restaurant she visited, you can say, “So tell me why you’re describing your restaurant experience.” You might find out that she didn’t hear your question correctly, or that she visited this restaurant before she went to a store to purchase her last computer.
If the participant misunderstood what you were asking her, reword or rephrase the task/question as needed to ensure that she understands what she is being asked to do. In a usability study, you can ask her to re-read the task out loud, and ask if she has any questions about the task. You may also need to provide some guidance on the type of feedback you’re looking for (e.g., functionality rather than visual design, thinking aloud, etc.) or point the participant toward an area of the product that she hasn’t found on her own.
While this pattern is fairly straightforward, it can be used in a wide range of situations to help keep the participant on the desired track. But be careful of its misuse or overuse; for example, if you’re prompting the participant to re-read a task every time she goes off the optimal or successful path for a task, she may catch on and it could start to influence her behavior.
See pattern in use
You can see this pattern being used in situations such as:
3.3 Redirect the participant
Sometimes the participant pursues a path that will be less useful to you, or the energy that she sends out doesn’t mesh with what you need from her. Use redirection to help align the participant with your goals.
When to use
Use this pattern when you need to take the participant’s focus and emotions and redirect them in a direction more appropriate for your study. This pattern may be appropriate:
If the participant seems fixated on something other than what you need to receive feedback on.
If the participant seems to be going down a path that will not be useful based on your study goals, and you want to bring her attention back to your original intent.
If you need to give the participant an assist so she is able to continue a task during a usability study.
If the participant is obviously disgruntled or unhappy with the organization/product, and you can tell that her complaints will get in the way of your feedback.
If the participant seems overly enthusiastic with the organization/product and loses focus.
What to do
This pattern redirects the participant toward an outcome more appropriate to the research you’re performing. The most common application of this pattern is the moderator staple of turning a question back around to the participant. For example, if she asks, “Where would I go to do that?” you may respond, “Where would you expect to go?” If the participant asks a question that is more difficult to turn around—for example, “Am I doing this right?—this pattern lets you provide a vague neutral answer that then redirects her to a topic more relevant to the study. For example, “Like I said, we’re definitely not testing you here. Let me ask you about.…”
You may also need to provide a usability study participant with an assist if she struggles with a task that you need to get feedback on, or if she is unable to complete a task that future tasks depend on. In these situations, you’re redirecting the participant from the path she is currently taking to the one that you need her to use so you can accomplish your study goals.
This pattern also lets you provide some space for the energy exuding from a disgruntled or overly enthusiastic participant. One way to do this is to, at the beginning of the session, let a disgruntled participant spend a limited amount of time venting to get the negativity out of her system. For example, if you’re interviewing her about how she decides to purchase a product and she immediately starts ranting about her experience on a specific e-commerce website, interrupt gently: “My goal for this session is to talk at a higher level about your decision making process. But since it seems like you’ve had a recent experience with this website, let’s take five minutes now for you to tell me about it, and then we’ll get back to my list of questions. Does that sound okay?” Once the participant has vented, tie her emotional state into what you’re asking her to do. For example, “I know you’ve had problems with this website in the past, and that is exactly why we’re doing this research—to understand how and why you use websites like this so we can find ways to make your experience better.”
See pattern in use
You can see this pattern being used in situations such as:
3.4 Reassure the participant
Sometimes the participant gets upset or needs to be comforted, either because of something happening inside the session or from something else going on in her life. Reassuring her can bring her emotional state to a more comfortable level so you can continue with the session.
When to use
Participants may become stressed or nervous about their performance during the session and need some gentle reassurance. This pattern may be appropriate:
What to do
Remind the participant that you’re not testing her in any way, and that you’re a neutral observer who wants her honest feedback, both positive and negative. If you didn’t design the product, you can let her know that as well. You can provide similar assurances during an interview or contextual inquiry to assure the participant that you’re a neutral party who is interested in her experience. If the participant needs additional reassurance, let her know that the feedback that she is providing is helpful and will help the team make the product better.
This pattern also includes reassuring the participant that she has control within the session to take a break if necessary and leave whenever she wants. Your reassurance should make her feel that you’ll do your best to make—and keep—her comfortable, even if you don’t explicitly say so. Remember your ethical obligations to the participant and do your best to ensure that she’ll leave the session feeling no worse—and ideally better—than she did when she arrived.
See pattern in use
You can see this pattern being used in situations such:
SHOULD YOU TELL A PARTICIPANT THAT HER FEEDBACK IS HELPFUL?
You may tell a participant that her feedback is the kind of feedback you’re looking for or that it is helpful. We feel it’s useful to provide this reassurance because it allows you to be more proactive than reactive about reassuring her.
Unfortunately, there are a few common misuses of this approach:
Feigned interest: If your tone sounds rote or machine-like, telling a participant that her feedback is good or helpful can come across as disingenuous.
Frequency: If you tell the participant that her feedback is good or helpful every time she says or does something, you start to sound like a broken record and she may lose trust in you. Similarly, providing this feedback at the wrong times could annoy her and possibly bias the study. For example, the participant may catch on if you only tell her that her feedback is useful when she says or does something in particular.
Wording: Saying “that is great feedback” or “that is good feedback” qualifies the participant’s feedback, which again may attune her to when you are or are not “praising” her feedback.
Be careful and methodical about how you provide any kind of affirmation. Instead of responding to everything that the participant says or does, pause periodically after a number of tasks or questions and tell her, “This is helpful for us to hear, and is the type of feedback we’re looking to get. Thank you. Let’s move on to.…” You only need to do this once or twice during a session, ideally toward the beginning of the session when participants are most guarded and can benefit from this feedback.
When providing any other feedback to the participant, stay neutral with responses such as “Okay,” or “Okay, thank you for that” instead of adding qualifiers. Additional guidelines for carefully using language and tone are discussed in section 2.5.
3.5 Build engagement
Sometimes a participant just doesn’t seem connected to what you’re asking her to do or talk about. By building engagement, you can help her feel more invested in the session. This emotional investment may make her more willing to provide the feedback you need.
When to use
Use this pattern to help draw an unengaged participant into the session. This may be appropriate if:
What to do
One way to build engagement is to adjust your body language, making sure that you’re taking the position of an active and interested listener (Figure 3.1). Sit up straight, angling your body slightly toward the participant. If you’re in the room with her, adjust your seat so you can maintain comfortable eye contact and she doesn’t have to look up or down at you. Keep your facial expression friendly and neutral. An occasional smile is okay, and you may notice the participant smiling back in return! Check your tone as well, remembering to sound genuinely interested in what she has to say.
Figure 3.1 Build engagement. A neutral moderating position (left), and an attempt to engage (right). © 2013 Mark Ainscow, used with permission.
If the participant seems disengaged with the product she is using, try to establish a personal connection between the participant and what she is doing. Encourage her to talk about herself or her experiences, and then tie her response into a rephrasing or refocusing of the task. For example, if you’re running a usability study on a mobile game that the participant seems to find uninteresting, ask her to tell you about mobile games that she does enjoy. After hearing her response, ask her to compare the experience of her favorite game to the one she is using as part of the study.
A bonus about making the personal connection is that the participant may forget about her initial emotional response (nervousness, etc.). But be careful to keep your efforts on building engagement linked to the session. This pattern is not about making friends with the participant; it’s about turning a topic or task that she finds boring, uninspiring, or irrelevant into something that she can provide feedback on.
In having this conversation with the participant, you may discover other reasons for the lack of engagement, such as low literacy or computer skills or a mismatch with your screening criteria. If this is the case, you may decide to shift the focus of the session to accommodate her unexpected skill level or criteria mismatch, or you may need to end the session early.
The Friendly Face and Inquisitive Mind moderating styles discussed in Chapter 2 are most appropriate to use during this pattern.
See pattern in use
You can see this pattern being used in situations such as:
3.6 Disengage from the participant
Sometimes a participant can get a little too personal or too enthusiastic in her interactions with you. Disengaging lets you politely pull away from that attention and establish a more neutral tone for the session.
When to use
This pattern may be appropriate:
If the participant is being overly chatty or is oversharing personal information.
If the participant is making you uncomfortable in any way, either through her attitude, something she is saying, or something she is doing.
Do not use this pattern if you feel that there is any danger presented by the participant’s behavior. Instead, take a break if you feel able to leave the room, and end the session early.
What to do
This pattern lets you gain some emotional distance from the participant (Figure 3.2). Move to a more formal tone of speaking, and reposition your body slightly away from her. Also try not to make too much eye contact with her, and focus more directly on the product or your notes. If you’re doing a contextual inquiry or usability study, you can also move yourself slightly behind the participant. This position makes it more difficult for her to directly interact with you and forces her to focus on the product. If you’re in a multiroom usability lab, you can also moderate from a different room. If you decide to switch to the other room in the middle of the session, simply say, “For the next few tasks, I’m going to be in the other room.”
Figure 3.2 Disengaging from the participant. A neutral moderating position (left), and an attempt to disengage (right). © 2013 Mark Ainscow, used with permission.
A variant of this pattern is to make a task or question less personal. This is useful if it seems that the personal nature of your question is causing the participant’s discomfort. For example, if you’re interviewing a participant about how she found a nursing home for her father but she is having a hard time talking about the process she went through, ask her instead to describe the last time she had to find a new doctor for herself. After she has talked for a bit about something more neutral and seems more comfortable, you can tie her response into your original personal question. For example, “You said that personal recommendations were extremely important in how you found your dermatologist. Tell me about if, and how, personal recommendations played a role in how you decided on a nursing home for your father?”
The Down to Business and By the Book moderating styles are most appropriate for use while applying this pattern.
See pattern in use
You can see this pattern being used in situations such as:
3.7 Take a break
Taking a break lets you hit the session’s “pause” button. This pause can reset the flow of the session and give you and the participant some time and space.
When to use
Use this pattern when you need to take some time, either for yourself or the participant. This pattern is appropriate:
What to do
Let the participant know that you’re taking a short break. If the reason you need to take a break won’t upset or offend the participant, you can provide that as a reason. For example, “The batteries on my audio recorder seem to have died. Let’s take a break for a few minutes while I replace them.” Alternatively, you can leave the reason vague, or stretch the truth if necessary. For example, “I need to check into something about the prototype.”
If you’re taking the break to give the participant space, offer to show her where the restroom is or get her something to drink. If you’re leaving her in the room and the recording equipment is still running, remind her of this so she doesn’t think she has some privacy and start making personal phone calls. Similarly, if the participant is remote and her screen is being shared, have her turn off the screen sharing first or remind her that her screen is still visible.
If possible, have a backup survey or questionnaire related to the study that you can give to the participant if you need to take a break and want to provide her with something useful to do. Section 15.2 describes this idea in more detail. Having this kind of backup also has the advantage of making the participant feel productive and useful—an advantage over leaving her staring nervously at herself in a one-way mirror!
See pattern in use
You can see this pattern being used in situations such as:
3.8 Shift the focus
Shifting the focus lets you adjust your study plan and research method to something that is a better fit for the participant’s qualifications or emotional state.
When to use
Use this pattern if you realize that your study plan is not going to work with the participant you have. This may happen if:
The participant is reacting poorly to the current form of research (e.g., because she says she would never perform the tasks you’re asking her to do and she can’t be persuaded to try anyway).
The participant doesn’t match your recruiting criteria, but is either high profile (e.g., a customer VIP) or if, based on your study goals or timing, it’s better to get some data than getting nothing.
Keep in mind that this pattern may not be appropriate if:
Your research goals are very narrowly defined and there is no value in adjusting the session (in which case, you should end the session early).
The participant doesn’t match your recruiting criteria and you have a backup/floater participant available who you could run instead (in which case, you should end the session early with your current participant).
What to do
Once you realize that your study plan isn’t working, decide if you can adjust the method to a form of research that better matches the participant’s qualifications or mental state. For example, if you’re running a usability study of a bank’s online account opening process and the participant says she’d only open an account at a branch, you might adjust your questions and tasks to more of an interview format to learn if she has opened an account at a branch before, what the experience was like, and how it could be improved. Or, if you’re interviewing a nurse about her interactions with patients but learn that she primarily deals with administrative tasks instead, you can change the focus from asking her questions to watching the workflow she has to follow to accomplish her tasks.
To be successful at this pattern, you must have a solid understanding of the research goals and how your findings are going to be used. This understanding is key in determining if this pattern is appropriate and, if so, how to best shift a session’s focus to yield useful feedback.
See pattern in use
You can see this pattern being used in situations such as:
3.9 End the session early
Ending a session early is your last resort when other strategies have not—or could not have– been successful.
When to use
If the participant is not responding to your efforts to make her more comfortable, or if external circumstances make it difficult or impossible to continue, you may need to end the session and cut your losses. This pattern may be appropriate:
If the participant is a misrecruit and there is no way to adjust the session to get feedback from her.
If the participant seems excessively upset or unable or unwilling to continue.
If you’re worried for your safety or that of the participant’s.
If technical or environmental issues make it difficult or impossible to continue.
What to do
In the case of technical or environmental issues, you have a built-in reason for ending the session. For example, if you have to evacuate your facility because of a fire, the intensity of the situation and unstable environmental conditions means you should let the participant leave.
Ending a session early for any other reason requires careful communication skills, as you don’t want the participant to think that you’re ending it because she did something wrong. If you did not provide her with a list of tasks or questions ahead of time, you can just skip any unfinished items and go straight to ending the session without any additional explanation. If she can tell that you’re skipping something, you can use a pretext such as, “The rest of these tasks/questions aren’t applicable to you/your role, so we’re going to skip them.” Other pretexts that may come in handy to explain a shortened session include:
“You went through everything faster than expected!”
“We’re having some technical difficulties, so rather than having you wait around, we’re going to end early so you can have time back in your day.”
The point at which you decide that a session is not worth saving—your breaking point—will be different for each study and the factors that are key to the study’s success. Think about it ahead of time and discuss it with your stakeholders: What are the deal-breakers that would make you end a session rather than continuing it?
See pattern in use
You can see this pattern being used in situations such as:
3.10 Choosing the best pattern for your situation
Now that you have all these patterns at your disposal, how do you decide which to use, and when? As you can see in Table 3.2, the patterns fit into the steps described in Chapter 2 for how to handle situations. Different patterns are more useful than others depending on what step you’re on.
As discussed in Chapter 2, you should always keep in mind your ethical obligations to ensure the comfort and safety of the participants, your organization, your team, and yourself. In addition to these obligations, consider the following when you decide what to do during a session, specifically, how hard you work to bring a session back on track instead of ending the session early:
1. Your research project’s goals. Think about what your stakeholders are trying to learn from this research, who they need to learn it from, and what will help them get the answers they need. For example, is the goal to discover issues, or to validate a product? How important are specific participant criteria? Being clear on these goals, and knowing that you and your stakeholders are on the same page about those goals, will let you make more efficient decisions.
2. Maintaining data quality. Based on your research goals, what sample size do you need? How many participants have you run already? How much consistency (of recruit criteria and of tasks/questions) do you need between participants? For example, if you’re running a qualitative usability study and have recruited twelve participants (assuming you’ll have two no-shows), you may be more willing to let the twelfth participant explore other areas of the product outside what’s in the study plan than when you’re running the second participant.
Because of all these factors, it’s impossible to spell out exactly which pattern is best to use in every situation. Instead, use these questions to identify what’s best for your situation.
We hope this chapter has given you an understanding of the tools available in your moderating Swiss Army knife and the key considerations that might affect how, when, and why, you’d use each of them. In Part 2, you’ll see these tools applied across a wide variety of situations.