RESEARCH METHOD
69 Remote Moderated Research
Remotely observing users completing tasks on their own electronic devices can reveal rich insights into contexts of use that cannot be replicated in a controlled lab environment.
Remote moderated research is a method adapted from traditional usability testing techniques, but relies on screen-sharing software in lieu of the usability lab equipment to conduct it. There is still live interaction between the researcher and participants in remote moderated tests, and anything from websites to prototypes, screen mock-ups, and sketches can be tested and evaluated. However, a key differentiator and benefit of the method is that it exposes rich, qualitative data about a participant’s native computer and possibly his or her physical environment, which usability tests that take place in a controlled lab setting do not.
Depending on your needs and time line, participants for remote moderated testing can be recruited using traditional means, or “live recruited” (see Time Aware Research). Live recruiting participants is particularly powerful, as the participant can be intercepted as he or she begins a process, and the research session can be initiated immediately upon his or her consent.1 This flexibility allows the research team to observe behavior in a task that the participant has selected, as opposed to the team assigning a task or set of tasks that may not carry a sense of urgency or importance to the user.
Once intercepted, observing how people complete tasks that they’ve initiated can be insightful. For instance, if your interface requires some organization of personal media (e.g., pictures, videos, or music), asking users to work with their own files can provide deeper insight into the organizing principles, tools, and workarounds people devise to create meaning and simplify access rather than assigning them to organize stock photos or other files to which they are not attached.
Unlike traditional usability testing, which requires that participants travel to the usability lab, remote moderated research opens up the opportunity to work with participants who are unable to travel because of geography or other limitations. As long as the participants have an Internet connection and a computer, under most circumstances the test can be successfully administered.2
Remote moderated research is not necessarily a cheaper option to the traditional usability test, nor is it likely that it will be completed in less time. Although you may be able to save on lab equipment costs, travel expenses, and even cut some costs related to recruiting, there are still the costs associated with participant incentives, and use of moderator and analyst time.3 Use remote moderated research when the benefits of accessing a geographically diverse group of participants, being able to live recruit participants, and studying people in their native environments outweigh the costs and time constraints.
1. Bolt, Nate, and Tony Tulathimutte. Remote Research: Real Users, Real Time, Real Research. San Francisco, CA: Rosenfeld Media, 2010.
2. See note 1 above.
3. See note 1 above.
Further Reading
Tullis, Tom, and Bill Albert. Measuring the User Experience. San Francisco, CA: Morgan Kaufmann, 2008.
Tullis, Tom, Donna Tedesco, and William Albert. Beyond the Usability Lab: Conducting Large-Scale User Experience Studies. San Francisco, CA: Morgan Kaufmann, 2010.
If you are recording sessions, it is imperative that you obtain consent from the participant, and disclose the ways in which the recording will be used. If you are recording a phone conversation separately from the screen capture software, certain state regulations may apply. Refer to the “Privacy and Consent” chapter in Remote Research for more information, and consult your legal team before you record any remote research sessions.
