RESEARCH METHOD
13 Cognitive Walkthrough
Cognitive walkthrough is a method that evaluates whether the order of cues and prompts in a system reflect the way people cognitively process tasks and anticipate “next steps” of a system.1
The cognitive walkthrough is a usability inspection method that evaluates a system’s relative ease-of-use in situations where preparatory instruction, coaching, or training of the system is unlikely to occur. In these situations—when a person must actively engage with an interface to know what to do next, rather than relying on preexisting knowledge of the system—each step of the interaction with the system can be assessed as a step that either moves the individual closer to or further from his goal. Cognitive walkthroughs provide a systematic way to identify these distinct points during an interaction sequence, and then evaluate whether each step is more likely to fail or succeed in helping people make the next correct decision in the interaction.2 Systems that meet these expectations are considered to be more usable and more learnable.
Cognitive walkthroughs are particularly well suited for evaluating “walk-up-and-use” systems that are primarily audio-or display-based—such as ATMs, automated parking garage or subway ticketing systems, and automated voice-response phone systems.3 A series of representative tasks should be selected, all written from the user’s vantage point, and outlined in a believable sequence of action steps. The method then sets out to critique each step in the action sequence and evaluate whether it is the right step at the right time. The success of the interface can be judged based on whether the system feedback either helps or hinders users to achieve their goals.4
The method’s focus on how people solve problems requires that evaluators ask the same four learning theory-based questions for each step in the action sequence.5
• Will users want to produce whatever effect the action has?
• Will users see the control (button, menu, label, etc.) for the action?
• Once users find the control, will they recognize that it will produce the effect that they want?
• After the action is taken, will users understand the feedback they get, so they can confidently continue on to the next action?
As the team evaluates each step in a task using the questions above, they will be able to make decisions about which sequence creates the fewest obstacles for the user. Because it cannot be assumed that users will be available for testing every step of the way along the iterative design process, expert reviews like cognitive walkthroughs ensure better use of participants’ time. However, because cognitive walkthroughs and usability testing tend to uncover different classes of design issues and usability problems, using them together—rather than in lieu of one another—is always recommended.
1. In the early 1990s, Peter Polson, Clayton Lewis, John Reiman, and Cathleen Wharton from the University of Colorado’s Institute of Cognitive Science introduced the cognitive walkthrough method. It was based on a theory of exploratory learning by Polson and Lewis, which is documented in their 1990 Human-Computer Interaction article “Theory-based Design for Easily Learned Interfaces.” See also:
Polson, Peter G., Clayton Lewis, John Rieman, and Cathleen Wharton. “Cognitive Walkthroughs: A Method for Theory-based Evaluation of User Interfaces.” International Journal of Man-Machine Studies 36, no. 5 (1992): 741–773.
2. Wharton, Cathleen, John Rieman, Clayton Lewis, and Peter Polson. “The Cognitive Walkthrough: A Practitioner’s Guide” in Usability Inspection Methods. New York: John Wiley and Sons, 1994.
3. See note 2 above.
4. See note 2 above.
5. See note 2 above.
Lewis, Clayton, and John Reiman. Task-centered User Interface Design: A Practical Introduction, 1993, http://www.hcibib.org
Cognitive walkthroughs are used to evaluate whether an interface is understandable and easy to learn based on the user’s problem-solving mental operations, and can be particularly effective in situations where a person is likely to be a first-or one-time user of the system.
The illustrated example shown to the below is based on an actual parking meter machine (see above), that clearly could have benefited from the cognitive walkthrough method.