Guidelines-based review

Draft

Introduction

In a guidelines-based review, designers or other somewhat knowledgeable persons are asked to check a system against a longer list of items. Guidelines can include hundreds of items ....

Guideline based reviews are done in several contexts:

• In usability testing, such reviews can represent either "low cost and better than nothing" reviews or be used as a complementary usability method.
• A related purpose is to test whether an application is consistent with organizational or platform-specific conventions, e.g. Mac OS X, Windows 7, Android, etc.

See also: heuristic evaluation, Research-Based Web Design & Usability Guidelines (US gov guidelines) and usability and user experience surveys

Method

Evaluators, e.g. designers or other persons (designers should not evaluate their own work) get a "sheet" of guidelines that can be annotated and rate a website for each of these criteria.

Ratings may be on a scale or just good/neutral/bad

Evaluators also should annotate each rating.

Some usability and user experience surveys also can be adopted/adapted/integrated for this kind of review.

List of well-known guidelines and manuals

We found most of these entries through a list in the excellent HCI bib.

Usability of web sites

Userfocus usability evaluation workbook

Authors: David Travis (userfocus.co.uk), July 6, 2009

Available both as web content and as free Excel workbook. Translations exist for several languages.

The web site and workbook are organized in 9 sections that include each between 13 and 37 guidelines. An evaluator then can rate a site +1/0/-1 for each guideline and add a comment. The worksheet will then compute percentages with respect to all filled-in ratings (some guidelines may not be relevant for your own web site). The 9 sections cover the following issues:

1. Home page usability: 20 guidelines to evaluate the usability of home pages.
2. Task orientation: 44 guidelines to evaluate how well a web site supports the users tasks.
3. Navigation and IA: 29 guidelines to evaluate navigation and information architecture.
4. Forms and data entry: 23 guidelines to evaluate forms and data entry.
5. Trust and credibility: 13 guidelines to evaluate trust and credibility.
6. Writing and content quality: 23 guidelines to evaluate writing and content quality.
7. Page layout and visual design: 38 guidelines to evaluate page layout and visual design.
8. Search usability: 20 guidelines to evaluate search.
9. Help, feedback and error tolerance: 37 guidelines

The workbook is locked to prevent mistakes, but you can unlock it and add/remove criteria.

Can it be used in education ? Undergraduate students of an introductory class did make sense out of most items. In addition, the workbook doesn't require that all items need to be used. Since learners were exposed to this workbook at the very beginning of the class, the workbook actually became an instrument for learning what web usability is about. - DKS, 28 April 2011

Usability.gov guidelines

Author: U.S. Department of Health & Human Services.

Available at usability.gov as one single or several PDF files, a format that actually presents fairly low usability with respect to on-line work ....

See also: Research-Based Web Design & Usability Guidelines for a list of some of these guidelines.

“The Research-Based Web Design & Usability Guidelines (Guidelines) were developed by the U.S. Department of Health and Human Services (HHS), in partnership with the U.S. General Services Administration. This new edition of the Guidelines updates the original set of 187 guidelines, and adds 22 new ones. Many of the guidelines were edited, and numerous new references have been added. There are now 209 guidelines. [..] The primary audiences for the Guidelines are Web site managers, designers, and others involved in the creation or maintenance of Web sites. A secondary audience is researchers who investigate Web design issues.” (p. XV)

Overall, we be believe that these currently are the best set of web usability guidelines - Daniel K. Schneider 16:48, 14 March 2011 (CET).

Each guideline is ranked in importance and scientific evidence. One foreword is written by Ben Shneiderman and we suspect that he played an important role in the design of this document. Authors were supported by a large panel of respected researchers and practitioners.

There is no printable checklist. However, in the appendices starting on page 205 you will find lists of guidelines ranked by relative importance and strength of evidence.

MIT Information services guidelines

Available at: Usability Guidelines

These guidelines are available as a single printable webpage, i.e. you can annotate each criteria with a pen.

Nielsen's Top ten Guidelines for Homepage Usability

Author: Jakob Nielsen (2002)

Available: Top Ten Guidelines for Homepage Usability at useit.com. (single printable HTML page)

Nielsen's 113 Guidelines for Homepage Usability

Author: Jakob Nielsen (2002)

Available: 113 Design Guidelines for Homepage Usability (single printable HTML page)

Other web design guidelines

The The Web Style Guide 3rd edition

Author: Patrick J. Lynch and Sarah Horton

The first version of the Web Style Guide was a web site called the Yale Web Style Guide and posted in 1993. A 2nd update was published in 1997 and in 1999 a first print edition was produced. The current, 3rd edition, is available both online (free and unabridged) and in book form.

This book isn't a guideline in the classical sense, rather an introductory text about web site design. However, several chapters include explicit guidelines, e.g. Universal Usability Guidelines.

NASA web Guidelines

(broken links, need to be repaired)

The guidelines are part of NASA's usability toolkit and that includes other interesting information, such as a usability method's overview.

Available as PDF document

These guidelines may not be very useful to everyone, but it shows how larger organizations work.

Intranet usability guidelines

Author: Tim Fidgeon (2006).

Available as a single printable web page: Intranet usability guidelines

Accessibility

See also web accessibility

W3C recommendations

• User Agent Accessibility Guidelines 1.0

• Web Content Accessibility Guidelines 1.0

• Checklist of Checkpoints for Web Content Accessibility Guidelines 1.0. This list is an appendix of the above web content guidelines. You can print it out and do checks with a pen.

508 guidelines

In 1998, the US Congress amended the Rehabilitation Act of 1973 (29 U.S.C. 794d) as amended by the Workforce Investment Act of 1998 (P.L. 105 - 220), August 7, 1998 to require Federal agencies to make their electronic and information technology (EIT) accessible to people with disabilities.

• Section508.gov (Official home page)
• WebAIM Section 508 Checklist. This is a summary checklist in one page (also as printable version).

Generic user interface guidelines

Shneiderman Eight Golden Rules of Interface Design

1. Strive for consistency.
2. Enable frequent users to use shortcuts.
3. Offer informative feedback.
4. Design dialog to yield closure.
5. Offer simple error handling.
6. Permit easy reversal of actions.
7. Support internal locus of control.
8. Reduce short-term memory load.

Read a summary produced at Washington University by Josh Tenenberg

Smith and Mosier, 1986

Available: Guidelines for designing user interface software, ESD-TR-86-278, August 1986, by Sidney L. Smith and Jane N. Mosier.

This is the oldest (very detailed) software interface guideline. It includes 944 (!) guidelines. According to Ben Shneiderman, Introduction to the Usability Guidelines Book, “These admirable efforts influenced many designers and contributed to the 1980s corporate design guidelines from Apple, Microsoft, and others covering personal computers, desktop environments, and public access kiosks.”

List of platform specific guidelines

• Introduction to Apple Human Interface Guidelines

• GNOME Human Interface Guidelines 2.2.1. Gnome is one of the desktops that are available for Linux. These guidelins include a few general usability principles, i.e.:
  1.  Design for People
  2.  Don't Limit Your User Base
  3.  Create a Match Between Your Application and the Real World
  4.  Make Your Application Consistent
  5.  Keep the User Informed
  6.  Keep It Simple and Pretty
  7.  Put the User in Control
  8.  Forgive the User
  9.  Provide Direct Manipulation

• Java Look and Feel Design Guidelines (version 2)

• There are more (of course). See for example Official Usability, User Experience & User Interface Guidelines From Companies by Justin Mifsud, Usability Geek, Published September 19, 2011

Humor

• Bad Usability Calendar, published yearly by Netlife Research, Norwegian UI/UX consultants

• Do you make these 4 mistakes when carrying out a usability review? (Userfocus.co.uk, Sept. 2011)

Bibliography

• Farkas, D.K. & Farkas, J.B. (2000). Guidelines for designing web navigation. Technical Communication, 47(3), 341-358.

• Lynch, Patrick, J and Sarah Horton (2009). Web Style Guide, 3rd edition: Basic Design Principles for Creating Web Sites, Yale University Press, ISBN 9780300137378.

• Mayhew, D. (1992). Principles and Guidelines in User Interface Design. Englewood Cliffs, NJ: Prentice-Hall.

• Nielsen, J. & Tahir, M. (2002). Homepage Usability: 50 Sites Deconstructed. Indianapolis, IN: New Riders Publishing.

• Park, I. & Hannafin, M.J. (1993). Empirically-based guidelines for the design of interactive multimedia. Educational Technology Research and Development, 41(3), 63-85.

• Ramey, J.A. (2000). Guidelines for Web data collection: Understanding and interacting with your users. Technical Communication, 47(3), 397-410.

• Shneiderman, Ben (2000). Universal usability. Communications of the ACM, 43(5), 84-91.

• Spyridakis, J.H. (2000). Guidelines for authoring comprehensible web pages and evaluating their success. Technical Communication, 47(3), 359-382.

• Williams, T.R. (2000). Guidelines for designing and evaluating the display of information on the Web. Technical Communication, 47(3), 383-396.