Course evaluation

Draft

Definition

Course evaluation ca take different forms:

• Formative evaluation by an expert
• Evaluation by students
• Self-evaluation that includes feedback from students

Types of evaluations

Kirkpatrick’s Four Levels of Evaluation (according to Terry Anderson, retrieved 18:15, 11 September 2006 (MEST))

1. Reaction is a measure of learners’ reactions.
2. Learning is a measure of what they learned.
3. Transfer is a measure of changes in their behaviour upon completion of the program.
4. Results is a measure of the business outcomes attributable to the learning which occurred in the program.
5. Return on investment (ROI) - added by Anderson.

Guidelines for evaluation

Merrill's first principles of instruction

• This framework is descibed in the Instructional design article

Dick and Carey (1996) guidelines

1. Are motivational concerns addressed?
2. Is the appropriate/relevant content included?
3. Is the presentation sequence of the content correct?
4. Is all of the required information available to the student?
5. Do appropriate and ample practice exercises exist?
6. Is adequate feedback included for these exercises?
7. Are appropriate tests provided to assess student progress?
8. Are sufficient follow through activities provided?
9. Is the student presented with a clear path/navigational guide to move them through the course material and components?
10. Are aides to assist the student with memorization and facilitate transfer of learning provided?

IPSRT

Baylor, Kitsanas, and Chung (2001) developed a similar reflective question tool called, Instructional Planning Self-Reflective Tool (IPSRT). The IPSRT is designed to facilitate self-reflective thinking through the lesson planning process for a traditional or online course.

Reeves and Hedberg

This model encompasses six functions or levels of evaluation that are keyed to the major stages involved in the design, development, and implementation of interactive learning systems or products such as multimedia DVD's, Web-based training, electronic performance support systems, and e-learning solutions.

Evaluation materials

Things to download

• Reeves and Hedberg forms and protocols

On-line tools

• IQAT - Interactive quality assessment tool - this is commercial $500/year benchmarking tool, demo available.

References

• Cooley, W. W., and Lohnes, P. R. (1976). Evaluation research in education. New York: Irvington.

• Dick, W., & Carey, L. (1996). The Systematic Design of Instruction, (4th Ed.). New York: Haper Collins College Publishers.

• Dick, W., Carey, L., & Carey, J. O., (2001). The systematic design of instruction (5th ed.). New York: Addison-Wesley, Longman.

• Flagg, B. N. (1990). Formative evaluation for educational technologies. Hillsdale, NJ: Lawrence Erlbaum.

• Kemp, J. E., Morrison, G. R., & Ross, S. M. (1998). Designing Effective Instruction, (2nd Ed.). Upper Saddle River, NJ: Prentice Hall.

• Kirkpatrick, D. (1979). Techniques for evaluating training programs. Training and Development Journal. 33(6), p. 78-92.

• Reeves, Thomas, C. and John G. Hedberg (2003), Interactive Learning Systems Evaluation, Englewood Cliffs: Educational Technology Publications. ISBN 0-87778-304-7. The companion web site makes available a full set of evaluation tools (forms and protocol)

• Scriven, M. (1993). Hard-won lessons in program evaluation. San Francisco, CA: Jossey-Bass.