COAP:COAP-2170/week2: Difference between revisions

The educational technology and digital learning wiki
Jump to navigation Jump to search
 
(22 intermediate revisions by the same user not shown)
Line 3: Line 3:


== COAP 2170 Week 2 program ==
== COAP 2170 Week 2 program ==
(provisional)


=== Monday - Usability testing and user satisfaction ===
=== Monday - Usability testing and user satisfaction ===
Line 10: Line 8:
; (1) Administration
; (1) Administration


* Catching up (new students(
* Catching up (new students)


; (2) Introduction to usability testing
; (2) Introduction to usability testing
Line 16: Line 14:
* [http://edutechwiki.unige.ch/mediawiki/images/1/10/ISO9241part11.png Picture of the ISO usability framework] (A global picture of usability)
* [http://edutechwiki.unige.ch/mediawiki/images/1/10/ISO9241part11.png Picture of the ISO usability framework] (A global picture of usability)


* Introductory videos and texts
* Introductory videos, comics and short texts
: [http://vimeo.com/12053758 Demo Usability Test by Steve Krug], Author of "Don't make me think!" (important introduction)
: [https://vimeo.com/51153773 Demo Usability Test by Steve Krug], Author of "Don't make me think!" (important introduction, core of demo starts at min 6:00, 1st task at min 8:15)
* Introductions for people who don't like much text
* [https://vimeo.com/search?q=%22steve+krug%22+usability More videos by Steve Krug] about usability.
: [http://www.userfocus.co.uk/articles/moderationcomic.html Usability Test Moderation: The Comic]. A copy is available [http://tecfa.unige.ch/guides/te/coap2170/comics/ here] (password protected). Probably also on the N: drive.
: [http://www.userfocus.co.uk/articles/moderationcomic.html Usability Test Moderation: The Comic]. A copy is available [http://tecfa.unige.ch/guides/te/coap2170/comics/ here] (password protected). Probably also on the N: drive.
: [http://sensible.com/Downloads/DMMTchapter09_for_personal_use_only.pdf Usability testing on 10 cents a day] by Steve Krug
: [http://sensible.com/Downloads/DMMTchapter09_for_personal_use_only.pdf Usability testing on 10 cents a day] by Steve Krug
Line 26: Line 24:
: [[Personas]] and their use in [[cognitive walkthrough]]s. We shall have a look at [http://wiki.fluidproject.org/display/fluid/Fluid+Personas at set of Personas]
: [[Personas]] and their use in [[cognitive walkthrough]]s. We shall have a look at [http://wiki.fluidproject.org/display/fluid/Fluid+Personas at set of Personas]
: [[usability testing]] (short overview of "classic" usability testing)
: [[usability testing]] (short overview of "classic" usability testing)
: [http://www.usability.gov/templates/docs/short_test_rep.doc Example of a short test] (Word, at usability.gov)
: Example of a short [http://www.usability.gov/sites/default/files/report-template-usability-test-short.docx Test Report] (Word, at usability.gov)
: Low-fi (low cost) vs. expensive detailed testing.
: Low-fi (low cost) vs. expensive detailed testing.
: Short presentation of [[heuristic evaluation]]
: Short presentation of [[heuristic evaluation]]
Line 34: Line 32:
: Download [http://www.userfocus.co.uk/resources/datalogger.html Usability Test Data Logger tool] (Excel file in a resource page) from [http://www.userfocus.co.uk/ UserFocus].
: Download [http://www.userfocus.co.uk/resources/datalogger.html Usability Test Data Logger tool] (Excel file in a resource page) from [http://www.userfocus.co.uk/ UserFocus].
:* Save the file in some appropriate place
:* Save the file in some appropriate place
:* On Windows, you will have to enable macros
:* On Windows, you will have to enable macros, otherwise configurations, menus, etc. will not work !
:* Start by filling in the first page of the excel workbook (click on all check boxes and in particular make sure that users are configured as they should). We will give a demo in the classroom.
:* Documentation: [http://www.userfocus.co.uk/pdf/DataloggerV5Manual.pdf Download the datalogger manual] (PDF)]
:* Documentation: [http://www.userfocus.co.uk/pdf/DataloggerV5Manual.pdf Download the datalogger manual] (PDF)]
 
<!--
; (3b) Livelogger demo (if I can manage to get it working)
; (3b) Livelogger demo (if I can manage to get it working)
: Please copy the folder from the N: drive/coap2170 (this software is not publicly available)
: Please copy the folder from the N: drive/coap2170 (this software is not publicly available)
Line 43: Line 42:


; (3c) Hands on usability testing
; (3c) Hands on usability testing
* Configuration of the Datalogger tool and collective task analysis walk-through example, e.g. obtaining information about web certificates (we shall discuss interesting tasks in class)
* Configuration of the Datalogger and collective task analysis walk-through example, e.g. obtaining information about a a study program.
* Idem for Livelogger
-->
: Let's have a look at http://www.webster.ac.th/ (Webster Thailand)
 
=== Wednesday - Usability testing (II) ===


; (4) Satisfaction questionnaires
; Satisfaction questionnaires
* We shall shortly look at the [[Usability_and_user_experience_surveys#System_Usability_Scale_-_SUS|SUS satisfaction questionnaire]]
* We shall shortly look at the [[Usability_and_user_experience_surveys#System_Usability_Scale_-_SUS|SUS system usability satisfaction]] and [[Usability_and_user_experience_surveys#TAM_Satisfaction_Questionnaire|technology acceptance]] questionnaires
: If time left, we will do with respect to Webster Geneva, using a online survey tool
: If time left, we will do it with respect to Webster web sites.


=== Wednesday - Usability testing (II) ===
'''Usability testing hands-on''':


; Usability testing hands-on
We shall do some lo-fi usability testing in pairs.  
We shall do some lo-fi usability testing in pairs.  


Line 59: Line 58:
* Task in one sentence
* Task in one sentence
* Text that you are going to read to the subject
* Text that you are going to read to the subject
* Explicit success criteria
* Explicit success criteria: How can you evaluate whether the user completed the task ?
 
(2) Student A will observe student B will do the task


(2) Student A will do and student B will observe. After a round of testing we will switch roles. Student A must not be familiar with the Website.
* After the first round (pilot study), adjust the tasks
* After one round of testing we will switch roles.  
* Students B must not be familiar with the Website.


* http://www.webster.ch (Webster Geneva)
{| class="wikitable"
* http://www.webster.ac.at/ (Webster Vienna)
|-
* http://www.webster.ac.th/ (Webster Thailand)
! Site !! Observer (A) !! Observed/testing subject (B)
* http://www.webster.nl/ (Webster Holland)
|-
* http://www.websterchina.com/ (Webster China)
| http://www.webster.ch (Webster Geneva) || Padmore - Kuhle ||
* http://www.bacl.ac.uk/ (Regent's American College, London)
|-
| http://www.webster.ac.at/ (Webster Vienna) || Oliver ||
|-
| http://www.webster.ac.th/ (Webster Thailand) || Anna ||
|-
| http://www.webster.nl/ (Webster Holland) || Toma ||
|-
|http://www.websterchina.com/ (Webster China) || Alec ||
|-
|http://www.webster.edu/ || Rea - Innocent ||
|-
|http://www.webster.edu/ghana || Evgeni - Adrien? ||
|-
|http://www.bacl.ac.uk/ (Regent's American College, London) || ||
|}


For each task, record something like:
For each task, record something like:
Line 76: Line 93:
* Your summary of the task testing
* Your summary of the task testing
* Your confidence in the results
* Your confidence in the results
Home page testing:
* You may design a special task related to the home page. E.g. ask the user if he/she understands what the site is about, what you can do with it, etc.
=== Software ===
* [http://www.userfocus.co.uk/resources/datalogger.html Usability Test Data Logger tool] (Excel file, see above)


== Homework 2 ==
== Homework 2 ==


* Read [http://sensible.com/Downloads/DMMTchapter09_for_personal_use_only.pdf Usability testing on 10 cents a day] by Steve Krug. Of course, you are welcome to explore more texts (including introductions in this wiki). But this little text is '''mandatory reading'''.
* Read [http://sensible.com/Downloads/DMMTchapter09_for_personal_use_only.pdf Usability testing on 10 cents a day] by Steve Krug. Of course, you are welcome to explore more texts (including introductions in this wiki). But this little text is '''mandatory reading'''.
* (If you missed class), read [http://www.usabilityfirst.com/usability-methods/usability-testing/ Usability testing] A short 1-page "how to"
* (If you missed class), read [http://www.usabilityfirst.com/usability-methods/usability-testing/ Usability testing] a short 1-page "how to" and also get inspiration from the US [http://www.usability.gov/sites/default/files/report-template-usability-test-short.docx Test Report]


Prepare a usability review (see also week 3)
Prepare a usability review (see also week 3)


* '''Either''' finish the usability test started in class on Wednesday '''or''' test Webster Geneva with a non-webster person (this is your choice, doing a analysis of Webster Geneva with a "foreign" person is a more interesting exercise)
* 'Finish the usability test started in class on Wednesday
* Do at least three tests/tasks.
* Do at least '''three''' tests/tasks with '''two''' users (plus one pilot)
* Make sure to produce meaningful data that could be used in a usability review
* Make sure to produce meaningful data that could be used in a usability review
* Upload the data files and/or the report to the worldclassroom as homework 2
* Upload the data files and/or the report to the worldclassroom as homework 2
* Use any data logging tool you like, but we suggest using either the ones presented in the classroom ''or'' taking notes on paper and then writing it up.
* Use any data logging tool you like, but we suggest using either the ones presented in the classroom ''or'' taking notes on paper and then writing it up.


On Monday week three we will meet a member of the marketing team and:
On '''Wednesday week three''' we will discuss both hw1 and hw2 outcomes and:
* Present highlights from hw1 (usability guidelines-based review of Webster Geneva)
* Present highlights from hw1 (usability guidelines-based review of Webster websites)
* Present highlights from homework 2
* Present highlights from homework 2
* Present the SUS survey results (if available) [[Usability_and_user_experience_surveys#System_Usability_Scale_-_SUS|SUS satisfaction questionnaire]]
* Discuss the TAM/SUS survey results (if available). See  [[Usability_and_user_experience_surveys#TAM_Satisfaction_Questionnaire|TAM]] and [[Usability_and_user_experience_surveys#System_Usability_Scale_-_SUS|SUS satisfaction]] questionnaires.
 
'''To fix in a new edition of this class'''
* Tell students to include (a) an introduction that justifies the tests plus (b) a conclusion that summarizes the results and discusses these.
 
== Usability in the gaming industry ==
 
For those who are interested in user experience research, computer games, live in the Geneva area, have nothing to do on Tuesday, speak french. The speaker is one of my ex-students at UniGe. You won't get extra points for attending ...


TECFA vous invite à son prochain séminaire, qui continue sur la lignée
'''Example tasks''' (for inspiration)
de la thématique des jeux vidéos/serious game, mardi 22 mars prochain de
12h15 à 13h45, à Uni Mail, salle 3393.


Notre intervenant sera Cyril Rebetez, qui a effectué sa thèse TECFA et
* Finding your class schedule ("Please find the schedule of COAP 2170")
obtenu son doctorat en psychologie en 2009. Il a depuis rejoint Sony
* Sign up to the mailing list in order to receive News (or "Find out if and how you can receive News")
Computer Entertainement Europe à Londres. A présent, il élabore des
* Tuition and fees
tests pour évaluer l'utilisabilité et l'expérience utilisateur des jeux
* This year's calendar
Playstation. Il a travaillé sur des jeux tels que Little Big Planet 2,
* Finding semester start dates
Start the Party, Killzone 3 ou encore MotorStorm: Apocalypse.
* Professor's email
* List of degrees
* Admission requirements
* Admission contacts
* Test the search function (please try to find "XY", e.g. graduate courses)
* Find housing offers
* Find out how to sign up on connections (and do it)


Le titre de son intervention est: "Introduction à l'utilisabilité des jeux vidéos"
'''Evaluation criteria'''
* Completeness (at least 3 tasks with 2 users)
* Quality of observations
* On time
* Overall quality


{{quotation|Nous commencerons par présenter l’intérêt que la recherche utilisateurs
If you aim for an A: Add an introduction that defines the objectives of the usability test and a conclusion that summarizes the results. I leave it open to you how you will present the results. An outstanding homework would be a complete (but small and focused) usability report (title, introduction, results, conclusion) that someone involved with the design of Webster web sites can understand. For data collection, you may use any tool you like (paper, Excel work sheet, data logger)
suscite pour l’industrie du jeu vidéos et pour les game designers en
particulier. Nous découvrirons comment Sony teste ses jeux autant du
point de vue pratique (objectifs, méthodes, analyse) que de
l’intégration dans des horaires de productions plutôt chargés. Nous
terminerons par quelques exemples d’études effectuées par notre équipe.}}

Latest revision as of 21:21, 20 January 2015

COAP 2170
COAP:COAP-2170
COAP:COAP-2170/term project
COAP:COAP-2170/week1
COAP:COAP-2170/week2
COAP:COAP-2170/week3
COAP:COAP-2170/week4
COAP:COAP-2170/week5
COAP:COAP-2170/week6
COAP:COAP-2170/week7
COAP:COAP-2170/week8

COAP 2170 Week 2 program

Monday - Usability testing and user satisfaction

(1) Administration
  • Catching up (new students)
(2) Introduction to usability testing
  • Introductory videos, comics and short texts
Demo Usability Test by Steve Krug, Author of "Don't make me think!" (important introduction, core of demo starts at min 6:00, 1st task at min 8:15)
Usability Test Moderation: The Comic. A copy is available here (password protected). Probably also on the N: drive.
Usability testing on 10 cents a day by Steve Krug
(3) Introduction to usability testing methodology and tools
Design and usability methods and techniques (FYI, overview page for various methods and tools)
Personas and their use in cognitive walkthroughs. We shall have a look at at set of Personas
usability testing (short overview of "classic" usability testing)
Example of a short Test Report (Word, at usability.gov)
Low-fi (low cost) vs. expensive detailed testing.
Short presentation of heuristic evaluation
(3a) Userfocus Datalogger user testing tool
This tool allows to simple task-based usability tests. In addition, you can administer a satisfaction questionnaire.
Download Usability Test Data Logger tool (Excel file in a resource page) from UserFocus.
  • Save the file in some appropriate place
  • On Windows, you will have to enable macros, otherwise configurations, menus, etc. will not work !
  • Start by filling in the first page of the excel workbook (click on all check boxes and in particular make sure that users are configured as they should). We will give a demo in the classroom.
  • Documentation: Download the datalogger manual (PDF)]

Wednesday - Usability testing (II)

Satisfaction questionnaires
If time left, we will do it with respect to Webster web sites.

Usability testing hands-on:

We shall do some lo-fi usability testing in pairs.

(1) Each student prepares a list of 3-5 tasks. Take a sheet of paper and write down:

  • Task in one sentence
  • Text that you are going to read to the subject
  • Explicit success criteria: How can you evaluate whether the user completed the task ?

(2) Student A will observe student B will do the task

  • After the first round (pilot study), adjust the tasks
  • After one round of testing we will switch roles.
  • Students B must not be familiar with the Website.
Site Observer (A) Observed/testing subject (B)
http://www.webster.ch (Webster Geneva) Padmore - Kuhle
http://www.webster.ac.at/ (Webster Vienna) Oliver
http://www.webster.ac.th/ (Webster Thailand) Anna
http://www.webster.nl/ (Webster Holland) Toma
http://www.websterchina.com/ (Webster China) Alec
http://www.webster.edu/ Rea - Innocent
http://www.webster.edu/ghana Evgeni - Adrien?
http://www.bacl.ac.uk/ (Regent's American College, London)

For each task, record something like:

  • Time it took
  • Success and failure: How easy/difficult was it ?
  • Important incidents and remarks made by the subject
  • Your summary of the task testing
  • Your confidence in the results

Home page testing:

  • You may design a special task related to the home page. E.g. ask the user if he/she understands what the site is about, what you can do with it, etc.

Software

Homework 2

Prepare a usability review (see also week 3)

  • 'Finish the usability test started in class on Wednesday
  • Do at least three tests/tasks with two users (plus one pilot)
  • Make sure to produce meaningful data that could be used in a usability review
  • Upload the data files and/or the report to the worldclassroom as homework 2
  • Use any data logging tool you like, but we suggest using either the ones presented in the classroom or taking notes on paper and then writing it up.

On Wednesday week three we will discuss both hw1 and hw2 outcomes and:

  • Present highlights from hw1 (usability guidelines-based review of Webster websites)
  • Present highlights from homework 2
  • Discuss the TAM/SUS survey results (if available). See TAM and SUS satisfaction questionnaires.

Example tasks (for inspiration)

  • Finding your class schedule ("Please find the schedule of COAP 2170")
  • Sign up to the mailing list in order to receive News (or "Find out if and how you can receive News")
  • Tuition and fees
  • This year's calendar
  • Finding semester start dates
  • Professor's email
  • List of degrees
  • Admission requirements
  • Admission contacts
  • Test the search function (please try to find "XY", e.g. graduate courses)
  • Find housing offers
  • Find out how to sign up on connections (and do it)

Evaluation criteria

  • Completeness (at least 3 tasks with 2 users)
  • Quality of observations
  • On time
  • Overall quality

If you aim for an A: Add an introduction that defines the objectives of the usability test and a conclusion that summarizes the results. I leave it open to you how you will present the results. An outstanding homework would be a complete (but small and focused) usability report (title, introduction, results, conclusion) that someone involved with the design of Webster web sites can understand. For data collection, you may use any tool you like (paper, Excel work sheet, data logger)

Retrieved from "https://edutechwiki.unige.ch/mediawiki/index.php?title=COAP:COAP-2170/week2&oldid=55039"