Course evaluation: Difference between revisions
Jump to navigation
Jump to search
(using an external editor) |
|||
| (13 intermediate revisions by 3 users not shown) | |||
| Line 3: | Line 3: | ||
== Definition == | == Definition == | ||
'''Course evaluation''' | '''Course evaluation''' can take different forms: | ||
* Formative evaluation by an expert | * Formative evaluation by an expert | ||
* Evaluation by students | * Evaluation by students | ||
* Self-evaluation that includes feedback from students | * Self-evaluation that includes feedback from students | ||
See also: [[e-benchmarking]] | |||
== Types of evaluations == | |||
Kirkpatrick’s Four Levels of Evaluation (according to [http://www.athabascau.ca/html/staff/academic/terrya/Effectiveness_of_E-learning_Presentation_files/v3_document.htm Terry Anderson], retrieved 18:15, 11 September 2006 (MEST)) | |||
# Reaction is a measure of learners’ reactions. | |||
# Learning is a measure of what they learned. | |||
# Transfer is a measure of changes in their behaviour upon completion of the program. | |||
# Results is a measure of the business outcomes attributable to the learning which occurred in the program. | |||
# Return on investment (ROI) - added by Anderson. | |||
== Guidelines for evaluation == | == Guidelines for evaluation == | ||
=== | === Merrill's first principles of instruction === | ||
* This framework is descibed in the [[Instructional_design#What_is_good_.22main-stream.22_Instructional_Design_.3F | Instructional design article]] | * This framework is descibed in the [[Instructional_design#What_is_good_.22main-stream.22_Instructional_Design_.3F | Instructional design article]] | ||
| Line 29: | Line 40: | ||
=== IPSRT === | === IPSRT === | ||
Baylor, Kitsanas, and Chung (2001) developed a similar reflective question tool called, Instructional Planning Self-Reflective Tool (IPSRT). The IPSRT is designed to facilitate self-reflective thinking through the lesson | Baylor, Kitsanas, and Chung (2001) developed a similar reflective question tool called, Instructional Planning Self-Reflective Tool (IPSRT). The IPSRT is designed to facilitate self-reflective thinking through the [[lesson planning]] process for a traditional or online course. | ||
=== Reeves and Hedberg === | === Reeves and Hedberg === | ||
This model encompasses six functions or levels of evaluation that are keyed to the major stages involved in the design, development, and implementation of interactive learning systems or products such as multimedia DVD's, Web-based training, electronic performance support systems, and e-learning solutions. | This model encompasses six functions or levels of evaluation that are keyed to the major stages involved in the design, development, and implementation of interactive learning systems or products such as multimedia DVD's, Web-based training, electronic performance support systems, and e-learning solutions. | ||
== Evaluation materials == | == Evaluation materials == | ||
| Line 45: | Line 55: | ||
* [http://www.iqat.org/tour.php IQAT - Interactive quality assessment tool] - this is commercial $500/year benchmarking tool, demo available. | * [http://www.iqat.org/tour.php IQAT - Interactive quality assessment tool] - this is commercial $500/year benchmarking tool, demo available. | ||
=== Lists for practionners === | |||
* [http://elearningindustry.com/a-compact-instructional-design-review-checklist A compact Instructional Design Review Checklist], elearningindustry.com (added 4/2013). | |||
* [http://blog.cathy-moore.com/2011/07/checklist-for-strong-elearning/ Checklist for strong elearning] | |||
5 July 2011 By Cathy Moore (added 4/2013). | |||
== References == | == References == | ||
* Cooley, W. W., and Lohnes, P. R. (1976). Evaluation research in education. New York: Irvington. | |||
* Dick, W., & Carey, L. (1996). The Systematic Design of Instruction, (4th Ed.). New York: Haper Collins College Publishers. | * Dick, W., & Carey, L. (1996). The Systematic Design of Instruction, (4th Ed.). New York: Haper Collins College Publishers. | ||
* Dick, W., Carey, L., & Carey, J. O., (2001). The systematic design of instruction (5th ed.). New York: Addison-Wesley, Longman. | * Dick, W., Carey, L., & Carey, J. O., (2001). The systematic design of instruction (5th ed.). New York: Addison-Wesley, Longman. | ||
* Flagg, B. N. (1990). Formative evaluation for educational technologies. Hillsdale, NJ: Lawrence Erlbaum. | |||
* Kemp, J. E., Morrison, G. R., & Ross, S. M. (1998). Designing Effective Instruction, (2nd Ed.). Upper Saddle River, NJ: Prentice Hall. | * Kemp, J. E., Morrison, G. R., & Ross, S. M. (1998). Designing Effective Instruction, (2nd Ed.). Upper Saddle River, NJ: Prentice Hall. | ||
* Kirkpatrick, D. (1979). Techniques for evaluating training programs. Training and Development Journal. 33(6), p. 78-92. | |||
* Naidu, Som (ed). [http://www.wikieducator.org/WikiEdProfessional_eLearning_Guidebook eLearning_Guidebook]. Commonwealth Education Media Center for Asia (CEMCA) and the Commonwealth of Learning (COL). There is a [http://www.wikieducator.org/WikiEdProfessional_eLearning_Guidebook/Evaluating_the_impacts_of_eLearning chapter on evaluation] | |||
* National Research Council (2003). Evaluating and improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics, [http://books.google.com/books?id=Xvx1xM6LD6UC GoogleBook link] | |||
* Reeves, Thomas, C. and John G. Hedberg (2003), Interactive Learning Systems Evaluation, Englewood Cliffs: Educational Technology Publications. ISBN 0-87778-304-7. The [http://www.evaluateitnow.com/ companion web site] makes available a full set of evaluation tools (forms and protocol) | * Reeves, Thomas, C. and John G. Hedberg (2003), Interactive Learning Systems Evaluation, Englewood Cliffs: Educational Technology Publications. ISBN 0-87778-304-7. The [http://www.evaluateitnow.com/ companion web site] makes available a full set of evaluation tools (forms and protocol) | ||
* Scriven, M. (1993). Hard-won lessons in program evaluation. San Francisco, CA: Jossey-Bass. | |||
[[Category: Design methodologies]] | [[Category: Design methodologies]] | ||
[[Category:Evaluation methods and grids]] | |||
[[Category: Program, course and teacher evaluation]] | |||
Latest revision as of 12:37, 16 April 2013
Definition
Course evaluation can take different forms:
- Formative evaluation by an expert
- Evaluation by students
- Self-evaluation that includes feedback from students
See also: e-benchmarking
Types of evaluations
Kirkpatrick’s Four Levels of Evaluation (according to Terry Anderson, retrieved 18:15, 11 September 2006 (MEST))
- Reaction is a measure of learners’ reactions.
- Learning is a measure of what they learned.
- Transfer is a measure of changes in their behaviour upon completion of the program.
- Results is a measure of the business outcomes attributable to the learning which occurred in the program.
- Return on investment (ROI) - added by Anderson.
Guidelines for evaluation
Merrill's first principles of instruction
- This framework is descibed in the Instructional design article
Dick and Carey (1996) guidelines
- Are motivational concerns addressed?
- Is the appropriate/relevant content included?
- Is the presentation sequence of the content correct?
- Is all of the required information available to the student?
- Do appropriate and ample practice exercises exist?
- Is adequate feedback included for these exercises?
- Are appropriate tests provided to assess student progress?
- Are sufficient follow through activities provided?
- Is the student presented with a clear path/navigational guide to move them through the course material and components?
- Are aides to assist the student with memorization and facilitate transfer of learning provided?
IPSRT
Baylor, Kitsanas, and Chung (2001) developed a similar reflective question tool called, Instructional Planning Self-Reflective Tool (IPSRT). The IPSRT is designed to facilitate self-reflective thinking through the lesson planning process for a traditional or online course.
Reeves and Hedberg
This model encompasses six functions or levels of evaluation that are keyed to the major stages involved in the design, development, and implementation of interactive learning systems or products such as multimedia DVD's, Web-based training, electronic performance support systems, and e-learning solutions.
Evaluation materials
Things to download
On-line tools
- IQAT - Interactive quality assessment tool - this is commercial $500/year benchmarking tool, demo available.
Lists for practionners
- A compact Instructional Design Review Checklist, elearningindustry.com (added 4/2013).
- Checklist for strong elearning
5 July 2011 By Cathy Moore (added 4/2013).
References
- Cooley, W. W., and Lohnes, P. R. (1976). Evaluation research in education. New York: Irvington.
- Dick, W., & Carey, L. (1996). The Systematic Design of Instruction, (4th Ed.). New York: Haper Collins College Publishers.
- Dick, W., Carey, L., & Carey, J. O., (2001). The systematic design of instruction (5th ed.). New York: Addison-Wesley, Longman.
- Flagg, B. N. (1990). Formative evaluation for educational technologies. Hillsdale, NJ: Lawrence Erlbaum.
- Kemp, J. E., Morrison, G. R., & Ross, S. M. (1998). Designing Effective Instruction, (2nd Ed.). Upper Saddle River, NJ: Prentice Hall.
- Kirkpatrick, D. (1979). Techniques for evaluating training programs. Training and Development Journal. 33(6), p. 78-92.
- Naidu, Som (ed). eLearning_Guidebook. Commonwealth Education Media Center for Asia (CEMCA) and the Commonwealth of Learning (COL). There is a chapter on evaluation
- National Research Council (2003). Evaluating and improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics, GoogleBook link
- Reeves, Thomas, C. and John G. Hedberg (2003), Interactive Learning Systems Evaluation, Englewood Cliffs: Educational Technology Publications. ISBN 0-87778-304-7. The companion web site makes available a full set of evaluation tools (forms and protocol)
- Scriven, M. (1993). Hard-won lessons in program evaluation. San Francisco, CA: Jossey-Bass.