RESEARCH METHOD
87 Think-aloud Protocol
Think-aloud protocol is a method that requires participants to verbalize what they are doing and thinking as they complete a task, revealing aspects of an interface that delight, confuse, and frustrate.1
Think-aloud protocol is among the most common evaluative methods in the usability community. The protocol is straightforward—it asks people to articulate what they are thinking, doing, or feeling as they complete a set of tasks that align with their realistic day-to-day goals. As a cornerstone method of the usability profession, it affords researchers with a tried-and-true approach not only to see the process of task completion unfold, but also to identify the aspects of a digital or physical product that delight, confuse, and frustrate people so that they can be corrected or improved. There are two common experimental procedures for the think-aloud protocol:
Concurrent Think-aloud is the most common way to conduct the method. The participant works through tasks while articulating what he or she is doing, thinking, and feeling. Depending on a myriad of factors ranging from the participant’s personality to task complexity, evaluators may have to repeatedly remind participants to verbalize what they are thinking as they work through a task. The focus of the test should be on what is happening, as opposed to why; people are reasonably able to speak about and complete a task at the same time without impacting the outcome of a task.2
Retrospective Think-aloud begins by asking participants to complete a task in silence (while their activity is recorded with video and/or a screen-capture device). Upon task completion, participants are invited to retrospectively comment on their processes as they watch a replay of their experience with a product or prototype. Retrospective think-alouds can provide additional insight into participant reasoning, intentions, and strategy.3
When planning a think-aloud session, rather than setting out to evaluate the usability of an entire product, focus efforts on evaluating aspects that can be tested independently—for instance, site navigation, or a single web form. Although the method is commonly conducted on either low-or high-fidelity prototypes, it can also be used to evaluate products already in the public domain such as competitor products or physical artifacts that require assembly (tents or children’s toys), syncing (GPS or MP3 devices), or customization (smartphones or body analysis and weight scales). Video and audio recordings can then be referred back to as testimony of how tasks are actually completed, as opposed to how the organization assumes they should be completed—a necessary shift when embracing a human-centered design philosophy.
1. The think-aloud protocol was adapted for use in the Human Computer Interaction community and documented by Clayton Lewis, an IBM researcher in Task-Centered User Interface Design: A Practical Introduction. The purpose of the protocol is to help researchers understand what aspects of the interface people are processing as they attempt to complete a specific task. Also see:
Newell, Albert, and Herbert A. Simon. Human Problem Solving. Englewood Cliffs, N.J.: Prentice Hall, 1972.
2. Ericsson, Anders, and Herbert A. Simon. Protocol Analysis: Verbal Reports as Data, Revised ed. Cambridge, MA: MIT Press, 1993.
3. Guan, Zhiwei, Shirley Lee, Elisabeth Cuddihy, and Judith Ramey. “The Validity of Stimulated Retrospective Think-Aloud Method as Measured by Eye Tracking.” CHI 2006 Conference Proceedings, 2006.
Further Reading
Dumas, Joseph S., and Janice C. Redish. A Practical Guide to Usability Testing. Exeter, England; Portland, OR: Intellect LTD, 1999.
Lewis, Clayton, and John Reiman. Task-Centered User Interface Design: A Practical Introduction. Boulder, CO: University of Boulder, Department of Computer Science, 1993.
Nielsen, Jakob. Usability Engineering. San Francisco, CA: Morgan Kaufmann, 1993.
CONDUCTING A THINK-A LOUD
Evaluators should ask participants to verbalize anything that they think, feel, do, or look at while processing the interface—essentially, to “think aloud” as they complete tasks. Evaluators can either ask that participants express their problem-solving approaches out loud as they go about the tasks (concurrent think aloud), or to complete the test in silence, and then review a video with the evaluator and describe their approach after the tasks are completed (retrospective think aloud). Think-alouds can be conducted on a range of low-fidelity to high-fidelity prototypes.
Courtesy of Eugene Eric Kim / Blue Oxen Associates
Courtesy of Kim Dowd