COAP:COAP-2170/week2: Difference between revisions

The educational technology and digital learning wiki
Jump to navigation Jump to search
 
(12 intermediate revisions by the same user not shown)
Line 3: Line 3:


== COAP 2170 Week 2 program ==
== COAP 2170 Week 2 program ==
(provisional)


=== Monday - Usability testing and user satisfaction ===
=== Monday - Usability testing and user satisfaction ===
Line 10: Line 8:
; (1) Administration
; (1) Administration


* Catching up (new students(
* Catching up (new students)


; (2) Introduction to usability testing
; (2) Introduction to usability testing
Line 16: Line 14:
* [http://edutechwiki.unige.ch/mediawiki/images/1/10/ISO9241part11.png Picture of the ISO usability framework] (A global picture of usability)
* [http://edutechwiki.unige.ch/mediawiki/images/1/10/ISO9241part11.png Picture of the ISO usability framework] (A global picture of usability)


* Introductory videos and texts
* Introductory videos, comics and short texts
: [http://vimeo.com/12053758 Demo Usability Test by Steve Krug], Author of "Don't make me think!" (important introduction)
: [https://vimeo.com/51153773 Demo Usability Test by Steve Krug], Author of "Don't make me think!" (important introduction, core of demo starts at min 6:00, 1st task at min 8:15)
* Introductions for people who like short texts
* [https://vimeo.com/search?q=%22steve+krug%22+usability More videos by Steve Krug] about usability.
: [http://www.userfocus.co.uk/articles/moderationcomic.html Usability Test Moderation: The Comic]. A copy is available [http://tecfa.unige.ch/guides/te/coap2170/comics/ here] (password protected). Probably also on the N: drive.
: [http://www.userfocus.co.uk/articles/moderationcomic.html Usability Test Moderation: The Comic]. A copy is available [http://tecfa.unige.ch/guides/te/coap2170/comics/ here] (password protected). Probably also on the N: drive.
: [http://sensible.com/Downloads/DMMTchapter09_for_personal_use_only.pdf Usability testing on 10 cents a day] by Steve Krug
: [http://sensible.com/Downloads/DMMTchapter09_for_personal_use_only.pdf Usability testing on 10 cents a day] by Steve Krug
Line 26: Line 24:
: [[Personas]] and their use in [[cognitive walkthrough]]s. We shall have a look at [http://wiki.fluidproject.org/display/fluid/Fluid+Personas at set of Personas]
: [[Personas]] and their use in [[cognitive walkthrough]]s. We shall have a look at [http://wiki.fluidproject.org/display/fluid/Fluid+Personas at set of Personas]
: [[usability testing]] (short overview of "classic" usability testing)
: [[usability testing]] (short overview of "classic" usability testing)
: [http://www.usability.gov/templates/docs/short_test_rep.doc Example of a short test] (Word, at usability.gov)
: Example of a short [http://www.usability.gov/sites/default/files/report-template-usability-test-short.docx Test Report] (Word, at usability.gov)
: Low-fi (low cost) vs. expensive detailed testing.
: Low-fi (low cost) vs. expensive detailed testing.
: Short presentation of [[heuristic evaluation]]
: Short presentation of [[heuristic evaluation]]
Line 37: Line 35:
:* Start by filling in the first page of the excel workbook (click on all check boxes and in particular make sure that users are configured as they should). We will give a demo in the classroom.
:* Start by filling in the first page of the excel workbook (click on all check boxes and in particular make sure that users are configured as they should). We will give a demo in the classroom.
:* Documentation: [http://www.userfocus.co.uk/pdf/DataloggerV5Manual.pdf Download the datalogger manual] (PDF)]
:* Documentation: [http://www.userfocus.co.uk/pdf/DataloggerV5Manual.pdf Download the datalogger manual] (PDF)]
 
<!--
; (3b) Livelogger demo (if I can manage to get it working)
; (3b) Livelogger demo (if I can manage to get it working)
: Please copy the folder from the N: drive/coap2170 (this software is not publicly available)
: Please copy the folder from the N: drive/coap2170 (this software is not publicly available)
Line 44: Line 42:


; (3c) Hands on usability testing
; (3c) Hands on usability testing
* Configuration of the Datalogger and/or the Livelogger tool and collective task analysis walk-through example, e.g. obtaining information about web.
* Configuration of the Datalogger and collective task analysis walk-through example, e.g. obtaining information about a a study program.
 
-->
=== Software ===
 
* [http://www.userfocus.co.uk/resources/datalogger.html Usability Test Data Logger tool] (Excel file in a resource page)
* Livelogger:copy the folder from the N: drive/coap2170


=== Wednesday - Usability testing (II) ===
=== Wednesday - Usability testing (II) ===


; Satisfaction questionnaires
; Satisfaction questionnaires
* We shall shortly look at the [[Usability_and_user_experience_surveys#System_Usability_Scale_-_SUS|SUS satisfaction questionnaire]]
* We shall shortly look at the [[Usability_and_user_experience_surveys#System_Usability_Scale_-_SUS|SUS system usability satisfaction]] and [[Usability_and_user_experience_surveys#TAM_Satisfaction_Questionnaire|technology acceptance]] questionnaires
: If time left, we will do it with respect to Webster web sites.
: If time left, we will do it with respect to Webster web sites.


Line 66: Line 60:
* Explicit success criteria: How can you evaluate whether the user completed the task ?
* Explicit success criteria: How can you evaluate whether the user completed the task ?


(2) Student A will do the task and student B will observe.
(2) Student A will observe student B will do the task


* After the first round (pilot study), adjust the tasks
* After the first round (pilot study), adjust the tasks
* After one round of testing we will switch roles.  
* After one round of testing we will switch roles.  
* Student A must not be familiar with the Website.
* Students B must not be familiar with the Website.


* http://www.webster.ch (Webster Geneva)
{| class="wikitable"
* http://www.webster.ac.at/ (Webster Vienna)
|-
* http://www.webster.ac.th/ (Webster Thailand)
! Site !! Observer (A) !! Observed/testing subject (B)
* http://www.webster.nl/ (Webster Holland)
|-
* http://www.websterchina.com/ (Webster China)
| http://www.webster.ch (Webster Geneva) || Padmore - Kuhle ||
* http://www.bacl.ac.uk/ (Regent's American College, London)
|-
| http://www.webster.ac.at/ (Webster Vienna) || Oliver ||
|-
| http://www.webster.ac.th/ (Webster Thailand) || Anna ||
|-
| http://www.webster.nl/ (Webster Holland) || Toma ||
|-
|http://www.websterchina.com/ (Webster China) || Alec ||
|-
|http://www.webster.edu/ || Rea - Innocent ||
|-
|http://www.webster.edu/ghana || Evgeni - Adrien? ||
|-
|http://www.bacl.ac.uk/ (Regent's American College, London) || ||
|}


For each task, record something like:
For each task, record something like:
Line 88: Line 96:
Home page testing:
Home page testing:
* You may design a special task related to the home page. E.g. ask the user if he/she understands what the site is about, what you can do with it, etc.
* You may design a special task related to the home page. E.g. ask the user if he/she understands what the site is about, what you can do with it, etc.
=== Software ===
* [http://www.userfocus.co.uk/resources/datalogger.html Usability Test Data Logger tool] (Excel file, see above)


== Homework 2 ==
== Homework 2 ==


* Read [http://sensible.com/Downloads/DMMTchapter09_for_personal_use_only.pdf Usability testing on 10 cents a day] by Steve Krug. Of course, you are welcome to explore more texts (including introductions in this wiki). But this little text is '''mandatory reading'''.
* Read [http://sensible.com/Downloads/DMMTchapter09_for_personal_use_only.pdf Usability testing on 10 cents a day] by Steve Krug. Of course, you are welcome to explore more texts (including introductions in this wiki). But this little text is '''mandatory reading'''.
* (If you missed class), read [http://www.usabilityfirst.com/usability-methods/usability-testing/ Usability testing] A short 1-page "how to"
* (If you missed class), read [http://www.usabilityfirst.com/usability-methods/usability-testing/ Usability testing] a short 1-page "how to" and also get inspiration from the US [http://www.usability.gov/sites/default/files/report-template-usability-test-short.docx Test Report]


Prepare a usability review (see also week 3)
Prepare a usability review (see also week 3)


* '''Either''' finish the usability test started in class on Wednesday '''or''' test Webster Geneva with a non-webster person (this is your choice, doing a analysis of Webster Geneva with a "foreign" person is a more interesting exercise)
* 'Finish the usability test started in class on Wednesday
* Do at least '''three''' tests/tasks with '''two''' users (plus one pilot)
* Do at least '''three''' tests/tasks with '''two''' users (plus one pilot)
* Make sure to produce meaningful data that could be used in a usability review
* Make sure to produce meaningful data that could be used in a usability review
Line 102: Line 114:
* Use any data logging tool you like, but we suggest using either the ones presented in the classroom ''or'' taking notes on paper and then writing it up.
* Use any data logging tool you like, but we suggest using either the ones presented in the classroom ''or'' taking notes on paper and then writing it up.


On '''Wednesday week three''' we will discuss both hw1 and hw outcomes and:
On '''Wednesday week three''' we will discuss both hw1 and hw2 outcomes and:
* Present highlights from hw1 (usability guidelines-based review of Webster Geneva)
* Present highlights from hw1 (usability guidelines-based review of Webster websites)
* Present highlights from homework 2
* Present highlights from homework 2
* Present the SUS/TAM survey results (if available) [[Usability_and_user_experience_surveys#System_Usability_Scale_-_SUS|SUS satisfaction questionnaire]]
* Discuss the TAM/SUS survey results (if available). See  [[Usability_and_user_experience_surveys#TAM_Satisfaction_Questionnaire|TAM]] and [[Usability_and_user_experience_surveys#System_Usability_Scale_-_SUS|SUS satisfaction]] questionnaires.


'''Example tasks''' (for inspiration)
'''Example tasks''' (for inspiration)

Latest revision as of 21:21, 20 January 2015

COAP 2170
COAP:COAP-2170
COAP:COAP-2170/term project
COAP:COAP-2170/week1
COAP:COAP-2170/week2
COAP:COAP-2170/week3
COAP:COAP-2170/week4
COAP:COAP-2170/week5
COAP:COAP-2170/week6
COAP:COAP-2170/week7
COAP:COAP-2170/week8

COAP 2170 Week 2 program

Monday - Usability testing and user satisfaction

(1) Administration
  • Catching up (new students)
(2) Introduction to usability testing
  • Introductory videos, comics and short texts
Demo Usability Test by Steve Krug, Author of "Don't make me think!" (important introduction, core of demo starts at min 6:00, 1st task at min 8:15)
Usability Test Moderation: The Comic. A copy is available here (password protected). Probably also on the N: drive.
Usability testing on 10 cents a day by Steve Krug
(3) Introduction to usability testing methodology and tools
Design and usability methods and techniques (FYI, overview page for various methods and tools)
Personas and their use in cognitive walkthroughs. We shall have a look at at set of Personas
usability testing (short overview of "classic" usability testing)
Example of a short Test Report (Word, at usability.gov)
Low-fi (low cost) vs. expensive detailed testing.
Short presentation of heuristic evaluation
(3a) Userfocus Datalogger user testing tool
This tool allows to simple task-based usability tests. In addition, you can administer a satisfaction questionnaire.
Download Usability Test Data Logger tool (Excel file in a resource page) from UserFocus.
  • Save the file in some appropriate place
  • On Windows, you will have to enable macros, otherwise configurations, menus, etc. will not work !
  • Start by filling in the first page of the excel workbook (click on all check boxes and in particular make sure that users are configured as they should). We will give a demo in the classroom.
  • Documentation: Download the datalogger manual (PDF)]

Wednesday - Usability testing (II)

Satisfaction questionnaires
If time left, we will do it with respect to Webster web sites.

Usability testing hands-on:

We shall do some lo-fi usability testing in pairs.

(1) Each student prepares a list of 3-5 tasks. Take a sheet of paper and write down:

  • Task in one sentence
  • Text that you are going to read to the subject
  • Explicit success criteria: How can you evaluate whether the user completed the task ?

(2) Student A will observe student B will do the task

  • After the first round (pilot study), adjust the tasks
  • After one round of testing we will switch roles.
  • Students B must not be familiar with the Website.
Site Observer (A) Observed/testing subject (B)
http://www.webster.ch (Webster Geneva) Padmore - Kuhle
http://www.webster.ac.at/ (Webster Vienna) Oliver
http://www.webster.ac.th/ (Webster Thailand) Anna
http://www.webster.nl/ (Webster Holland) Toma
http://www.websterchina.com/ (Webster China) Alec
http://www.webster.edu/ Rea - Innocent
http://www.webster.edu/ghana Evgeni - Adrien?
http://www.bacl.ac.uk/ (Regent's American College, London)

For each task, record something like:

  • Time it took
  • Success and failure: How easy/difficult was it ?
  • Important incidents and remarks made by the subject
  • Your summary of the task testing
  • Your confidence in the results

Home page testing:

  • You may design a special task related to the home page. E.g. ask the user if he/she understands what the site is about, what you can do with it, etc.

Software

Homework 2

Prepare a usability review (see also week 3)

  • 'Finish the usability test started in class on Wednesday
  • Do at least three tests/tasks with two users (plus one pilot)
  • Make sure to produce meaningful data that could be used in a usability review
  • Upload the data files and/or the report to the worldclassroom as homework 2
  • Use any data logging tool you like, but we suggest using either the ones presented in the classroom or taking notes on paper and then writing it up.

On Wednesday week three we will discuss both hw1 and hw2 outcomes and:

  • Present highlights from hw1 (usability guidelines-based review of Webster websites)
  • Present highlights from homework 2
  • Discuss the TAM/SUS survey results (if available). See TAM and SUS satisfaction questionnaires.

Example tasks (for inspiration)

  • Finding your class schedule ("Please find the schedule of COAP 2170")
  • Sign up to the mailing list in order to receive News (or "Find out if and how you can receive News")
  • Tuition and fees
  • This year's calendar
  • Finding semester start dates
  • Professor's email
  • List of degrees
  • Admission requirements
  • Admission contacts
  • Test the search function (please try to find "XY", e.g. graduate courses)
  • Find housing offers
  • Find out how to sign up on connections (and do it)

Evaluation criteria

  • Completeness (at least 3 tasks with 2 users)
  • Quality of observations
  • On time
  • Overall quality

If you aim for an A: Add an introduction that defines the objectives of the usability test and a conclusion that summarizes the results. I leave it open to you how you will present the results. An outstanding homework would be a complete (but small and focused) usability report (title, introduction, results, conclusion) that someone involved with the design of Webster web sites can understand. For data collection, you may use any tool you like (paper, Excel work sheet, data logger)

Retrieved from "https://edutechwiki.unige.ch/mediawiki/index.php?title=COAP:COAP-2170/week2&oldid=55039"