Cognitive walkthrough

Draft

<pageby nominor="false" comments="false"/>

Introduction

According to the NASA usability toolkit, retrieved 12:36, 18 March 2011 (CET), A cognitive walkthroug, is a “process of going step by step through a product or system design and getting reactions from key team players and typical users. One or two members of the design team can guide the walk through while one or more users will comment as the walk through proceeds.”

The Fluid project, retrieved 12:36, 18 March 2011 (CET), provides the following definition: “A cognitive walkthrough is a step-by-step exploration of a service to see how well a particular type of user, usually represented by a persona, is able to accomplish a particular objective or set of objectives. The objectives selected for testing are dictated by the persona that is chosen.”

James Hom defines cognitive walthrough as “review technique where expert evaluators construct task scenarios from a specification or early prototype and then role play the part of a user working with that interface--"walking through" the interface. They act as if the interface was actually built and they (in the role of a typical user) was working through the tasks. Each step the user would take is scrutinized: impasses where the interface blocks the "user" from completing the task indicate that the interface is missing something. Convoluted, circuitous paths through function sequences indicate that the interface needs a new function that simplifies the task and collapses the function sequence.”

Methods

Fluid project overview

The Fluid project roughly suggest the following procedure.

(1) Choose a user from whose perspective the walkthrough will be done. You may choose a persona for that.

(2) Define what the person wants to achieve

(3) Define the steps that this person should do in order to achive her/his goals

(4) Perform the task and take notes about the following kind of issues for each step:

• Will the user know what to do at this step?
• Is complex problem solving needed to figure out what to do?
• Will they know that they did the right thing (if they manage) and are making progress towards their goal?
• Is complex problem solving needed to interpret the feedback?

This project also defines a much more elaborated UX Walkthrough Process. In particular, it suggests to formalize the inspection process along three axis:

1. A protocol that clearly specifies what we are going to do, and what information we are going to capture along the way.
2. A predetermined clearly specified target that we are going to inspect: (product, version, instance, set of chunks) - the thing we're going to do it to.
3. A report template specifying the format, style and content of the report of the information we capture.

(retrieved 12:36, 18 March 2011 (CET))

Look at their Generic UX Walkthrough Report Template if you need an example

Spencer, 2000

Spencer (2000), presents the following model for a streamlined Cognitive Walkthrough process, adapted directly from Wharton, et al. (1994).

1. Define inputs to the walkthrough
   1. Identification of users
   2. Sample tasks for evaluation
   3. Action sequences for completing the tasks
   4. Description or implementation of interface
2. Convene the walkthrough
   1. Describe the goals of the walkthrough
   2. Describe what will be done during the CW
   3. Describe what will not be done during the
   4. lkthrough
   5. Explicitly defuse defensiveness
   6. Post ground rules in a visible place
   7. Assign roles
   8. Appeal for submission to leadership
3. Walkthrough the action sequences for each task
   1. Tell a credible story for these two questions:
      • Will the user know what to do at this step?
      • If the user does the right thing, will they know that they did the right thing, and are making progress towards their goal?
   2. Maintain control of the CW, enforce the 'ground rules (see below)
4. Record critical information
   1. Possible learnability problems
   2. Design ideas
   3. Design gaps
   4. Problems in the Task Analysis
5. Revise the interface to fix the problems

Implementing a cognitive walkthrough procedure is not obvious. Spencer (2000) claims that in certain situation a stream-lined version would be more successful. In particular he firstly suggests to adopt the following ground rules:

• Avoid design discussions (no designing, no defining of a design)
• No debating of cognitive theory
• The usability specialist is the leader of the session

Second, he argues in favor of simplication, in particular with respect to Warton:

• Collapse/simplify Warton's walk-through questions.
• Capturing Less Data

According to results of a study he conducted, {{quotation|Streamlining the walkthrough may trade-off granularity for coverage, but without that trade off, program managers and developers may perceive the walkthrough as being an inefficient use of time. Performing a streamlined CW is a good way to profile a user interface for potential problem areas, identify many steps that may be problematic for users, and accurately predict many usability problems. However the method will probably result in a few false positives.

Links

Introductions

• Cognitive Walkthrough at Fluid. Added by Jonathan Hung, last edited by Allison Bloodworth on May 26, 2009.

• UX Walkthrough Process at Fluid Added by Colin Clark, last edited by Paul Zablosky on Feb 12, 2009

• Cognitive Walkthrough by James Hom.

Examples

• Fluid project makes available a series of well-documented examples (from real studies). Search for walkthrough in order to retrieve the impressive list of shared documents. Below are just 2 examples plus one of the templates:
  • uPortal Walkthrough - Ed (Student) Persona (Dogle)
  • Sakai - Justin's Walkthrough
  • Generic UX Walkthrough Report Template

Bibliography

John, B. E, & Packer, H. Learning and using the cognitive walkthrough method: A case study approach, in Proceedings of CHI '95 (Denver CO, May 1995), ACM Press, 429-436.

Rowley, D. E., and Rhoades, D. G. The cognitive jogthrough: A fast-paced user interface evaluation procedure. Proceedings of CH1 '92 (May 1992), ACM Press, 389-395.

• Spencer, Rick. (2000). The streamlined cognitive walkthrough method" CHI 2000 Proceedings, 353-359. PDF at ACM, PDF at Fluid

• Wharton, Cathleen, et. al. (1994). "The Cognitive Walkthrough Method: A Practictioner's Guide." in Nielsen, Jakob, and Mack, R. (eds), Usability Inspection Methods, New York: John Wiley, ISBN 0-471-01877-5.