Evidence-centered design

Draft

Introduction

Evidence-centered Design is both a design and evaluation method.

According to Shute (2010:139), the ECD process “begins by identifying what should be assessed in terms of knowledge, skills, or other learner attributes. These variables cannot be observed directly, so behaviors and performances that demonstrate these variables need to be identified instead. The next step is determining the types of tasks or situations that would draw out such behaviors or performances.”

Evidenced-centered design and evaluation is particular important for designs that stress higher-level applicable knowledge. That could include interactive software like so-called "epistemic games", i.e. “games designed to give learners the rich experience of professional practica within a discipline. They serve to develop domain-specific expertise based on principles of collaborative learning, distributed expertise, and complex problem-solving.” (Rupp et al. 2010).

Bibliography

• Behrens, John T.; Robert J. Mislevy, Malcolm Bauer, David M. Williamson, Roy Levy. 2004. Introduction to Evidence Centered Design and Lessons Learned From Its Application in a Global E-Learning Program, International Journal of Testing, Vol. 4, Iss. 4. [

• Mislevy, R. J. (1994). Evidence and inference in educational assessment, Psychometrika, 59(4), 439–483. Abstract/PDF (Access restricted)

• Mislevy, R. J., & Haertel, G. D. (2006). Implications of evidence-centered design for educational testing. Educational Measurement: Issues and Practice, 25(4), 6–20.

• Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2004). A Brief Introduction to Evidence-Centered Design (CSE Report 632). CA: Center for Research on Evaluation, Standards, and Student Testing. (ERIC Document Reproduction Service No. ED483399)

• Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1(1), 3–62.

• Mislevy, R.J. (2006). Cognitive psychology and educational assessment. In R. L. Brennan (Ed.), Educational measurement (4th edition) (pp. 257–305). Portsmouth, NH: Greenwood Publishing Group.

• Mislevy, R.J., Steinberg, L.S., Almond, R.G., & Lukas, J.F. (2006). Concepts, terminology, and basic models of evidence-centered design. In D. M. Williamson, I. I. Bejar, & R. J. Mislevy (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 15–48). Mahwah, NJ: Erlbaum.

• Rupp, André A.; Matthew Gushta, Robert J Mislevy, David Williamson Shaffer (2010). Evidence-centered Design of Epistemic Games: Measurement Principles for Complex Learning Environments, JTLA, Vol 8, No 4. (Abstract/PDF), (Open access)

• Shute, V. J., Masduki, I., & Donmez, O. (2010). Conceptual Framework for Modeling, Assessing and Supporting Competencies within Game Environments. Technology Instruction Cognition and Learning, 8(2), 137–161. Retrieved from http://myweb.fsu.edu/vshute/pdf/TICL2010.pdf