COAP:COAP-2170/week2
Jump to navigation
Jump to search
COAP 2170 Week 2 program
Monday - Usability testing and user satisfaction
- (1) Administration
- Catching up (new students)
- (2) Introduction to usability testing
- Picture of the ISO usability framework (A global picture of usability)
- Introductory videos, comics and short texts
- Demo Usability Test by Steve Krug, Author of "Don't make me think!" (important introduction, core of demo starts at min 6:00, 1st task at min 8:15)
- More videos by Steve Krug about usability.
- Usability Test Moderation: The Comic. A copy is available here (password protected). Probably also on the N: drive.
- Usability testing on 10 cents a day by Steve Krug
- (3) Introduction to usability testing methodology and tools
- Design and usability methods and techniques (FYI, overview page for various methods and tools)
- Personas and their use in cognitive walkthroughs. We shall have a look at at set of Personas
- usability testing (short overview of "classic" usability testing)
- Example of a short Test Report (Word, at usability.gov)
- Low-fi (low cost) vs. expensive detailed testing.
- Short presentation of heuristic evaluation
- (3a) Userfocus Datalogger user testing tool
- This tool allows to simple task-based usability tests. In addition, you can administer a satisfaction questionnaire.
- Download Usability Test Data Logger tool (Excel file in a resource page) from UserFocus.
- Save the file in some appropriate place
- On Windows, you will have to enable macros, otherwise configurations, menus, etc. will not work !
- Start by filling in the first page of the excel workbook (click on all check boxes and in particular make sure that users are configured as they should). We will give a demo in the classroom.
- Documentation: Download the datalogger manual (PDF)]
Wednesday - Usability testing (II)
- Satisfaction questionnaires
- We shall shortly look at the SUS system usability satisfaction and technology acceptance questionnaires
- If time left, we will do it with respect to Webster web sites.
Usability testing hands-on:
We shall do some lo-fi usability testing in pairs.
(1) Each student prepares a list of 3-5 tasks. Take a sheet of paper and write down:
- Task in one sentence
- Text that you are going to read to the subject
- Explicit success criteria: How can you evaluate whether the user completed the task ?
(2) Student A will observe student B will do the task
- After the first round (pilot study), adjust the tasks
- After one round of testing we will switch roles.
- Students B must not be familiar with the Website.
| Site | Observer (A) | Observed/testing subject (B) |
|---|---|---|
| http://www.webster.ch (Webster Geneva) | Padmore - Kuhle | |
| http://www.webster.ac.at/ (Webster Vienna) | Oliver | |
| http://www.webster.ac.th/ (Webster Thailand) | Anna | |
| http://www.webster.nl/ (Webster Holland) | Toma | |
| http://www.websterchina.com/ (Webster China) | Alec | |
| http://www.webster.edu/ | Rea - Innocent | |
| http://www.webster.edu/ghana | Evgeni - Adrien? | |
| http://www.bacl.ac.uk/ (Regent's American College, London) |
For each task, record something like:
- Time it took
- Success and failure: How easy/difficult was it ?
- Important incidents and remarks made by the subject
- Your summary of the task testing
- Your confidence in the results
Home page testing:
- You may design a special task related to the home page. E.g. ask the user if he/she understands what the site is about, what you can do with it, etc.
Software
- Usability Test Data Logger tool (Excel file, see above)
Homework 2
- Read Usability testing on 10 cents a day by Steve Krug. Of course, you are welcome to explore more texts (including introductions in this wiki). But this little text is mandatory reading.
- (If you missed class), read Usability testing a short 1-page "how to" and also get inspiration from the US Test Report
Prepare a usability review (see also week 3)
- 'Finish the usability test started in class on Wednesday
- Do at least three tests/tasks with two users (plus one pilot)
- Make sure to produce meaningful data that could be used in a usability review
- Upload the data files and/or the report to the worldclassroom as homework 2
- Use any data logging tool you like, but we suggest using either the ones presented in the classroom or taking notes on paper and then writing it up.
On Wednesday week three we will discuss both hw1 and hw2 outcomes and:
- Present highlights from hw1 (usability guidelines-based review of Webster websites)
- Present highlights from homework 2
- Discuss the TAM/SUS survey results (if available). See TAM and SUS satisfaction questionnaires.
Example tasks (for inspiration)
- Finding your class schedule ("Please find the schedule of COAP 2170")
- Sign up to the mailing list in order to receive News (or "Find out if and how you can receive News")
- Tuition and fees
- This year's calendar
- Finding semester start dates
- Professor's email
- List of degrees
- Admission requirements
- Admission contacts
- Test the search function (please try to find "XY", e.g. graduate courses)
- Find housing offers
- Find out how to sign up on connections (and do it)
Evaluation criteria
- Completeness (at least 3 tasks with 2 users)
- Quality of observations
- On time
- Overall quality
If you aim for an A: Add an introduction that defines the objectives of the usability test and a conclusion that summarizes the results. I leave it open to you how you will present the results. An outstanding homework would be a complete (but small and focused) usability report (title, introduction, results, conclusion) that someone involved with the design of Webster web sites can understand. For data collection, you may use any tool you like (paper, Excel work sheet, data logger)