Educational technology research approaches

Draft

--- by NO means of real interest for the moment !!

Introduction

This article will aim to give an overview on different research approaches popular in educational technology and also present some of the debates.

Research approaches

What do journals require ?

The Journal of the Learning Sciences

(retrieved 23:00, 12 September 2006 (MEST))

• is not very explicit:

The Journal of the Learning Sciences (JLS) provides a multidisciplinary forum for the presentation of research on learning and education. It seeks to foster new ways of thinking about learning that will allow our understanding of cognition and social cognition to have impact in education. JLS publishes research articles that advance our understanding of learning in real-world situations and of promoting learning in such venues, including articles that report on the roles technology can play in promoting deep and lasting learning and in promoting engaged and thoughtful participation in learning activities, and articles reporting on new methodologies that enable rigorous investigation of learning in real-world situations. Scope

Educational Technology Research and Development

(retrieved 23:00, 12 September 2006 (MEST))

The Research Section assigns highest priority in reviewing manuscripts to rigorous original quantitative, qualitative, or mixed methods studies on topics relating to applications of technology or instructional design in educational settings. Such contexts include K-12, higher education, and adult learning (e.g., in corporate training settings). Analytical papers that evaluate important research issues related to educational technology research and reviews of the literature on similar topics are also published. This section features well documented articles on the practical aspects of research as well as applied theory in educational practice and provides a comprehensive source of current research information in instructional technology.

The Development Section publishes research on planning, implementation, evaluation and management of a variety of instructional technologies and learning environments. Empirically-based formative evaluations and theoretically-based instructional design research papers are welcome, as are papers that report outcomes of innovative approaches in applying technology to instructional development. Papers for the Development section may involve a variety of research methods and should focus on one or more aspect of the instructional development process; when relevant and possible, papers should discuss the implications of instructional design decisions and provide evidence linking outcomes to those decisions. See: About his Journal

Journal of Interactive Learning Research

(retrieved 23:00, 12 September 2006 (MEST)) This journal is most explicit and the The Scope and Standards of the Journal of Interactive Learning Research is signed Thomas C. Reeves.

Many researchers fail to distinguish clearly between the goals of their research and the methods they employ. Figures 1 and 2 present a classification scheme intended to distinguish between the goals and the methods of research. Most research studies submitted to JILR should be able to be classified according to the six research goals represented in Figure 1. This scheme reflects the debate about research "paradigms" that has dominated social science research literature for decades. For example, Soltis (1992) claims there are currently "three major paradigms, or three different ways of investigating important aspects of education" (p. 620): 1) the positivist or quantitative paradigm, 2) the interpretivist or qualitative paradigm, and 3) the critical theory or neomarxist paradigm. The three categories presented by Soltis (1992) fail to capture the full breadth of research goals in the fields of inquiry relevant to JILR, and therefore the scheme in Figure 1 includes more categories. However, the goals of inquiry represented in Figure 1 are not intended to be a complete and final listing of research goals.

A methodology classification scheme is represented in Figure 2. There are numerous methods available to researchers in areas as diverse as cognitive psychology, instructional technology, and computer science (cf., Driscoll, 1995), but for the sake of simplicity, these five methodological groupings provide sufficient discrimination to represent the major approaches likely to be used in investigations reported in JILR. This journal will be especially open to submissions that involve alternative methods (e.g., qualitative and critical theory) which seem to be underrepresented in more traditional publications.

Scope

See below for Figure 1 and Figure 2. This journal accepts a very broad spectrum but requires both rigor and responsibility of course.

Thomas C. Reeves Journal of Interactive Learning Research classification

Theoretical	research focused on explaining phenomena through the logical analysis and synthesis of theories, principles, and the results of other forms of research such as empirical studies.
Empirical	research focused on determining how education works by testing conclusions related to theories of communication, learning, performance, and technology.
Interpretivist	research focused on portraying how education works by describing and interpreting phenomena related to human communication, learning, performance, and the use of technology.
Postmodern	research focused on examining the assumptions underlying applications of technology in human communication, learning, and performance with the ultimate goal of revealing hidden agendas and empowering disenfranchised minorities.
Developmental	research focused on the invention and improvement of creative approaches to enhancing human communication, learning, and performance through the use of technology and theory.
Evaluation	research focused on a particular program, product, or method, usually in an applied setting, for the purpose of describing it, improving it, or estimating its effectiveness and worth.

Figure 1. Research goal classification scheme

Quantitative	experimental, quasi-experimental, correlational, and other methods that primarily involve the collection of quantitative data and its analysis using inferential statistics.
Qualitative	observation, case-studies, diaries, interviews, and other methods that primarily involve the collection of qualitative data and its analysis using grounded theory and ethnographic approaches.
Critical Theory	deconstruction of "texts" and the technologies that deliver them through the search for binary oppositions, hidden agendas, and the disenfranchisement of minorities.
Literature Review	various forms of research synthesis that primarily involve the analysis and integration of other forms of research, e.g., frequency counts and meta-analyses.
Mixed-methods	research approaches that combine a mixture of methods, usually quantitative and qualitative, to triangulate findings.

Figure 2. Research methods classification scheme

Some debates

Lack of quality

Research comes under fire from very different angles

From evaluation research (in distance education [1] but also present in educational technology): ssues raised include:

• lack of experimental control
• lack of procedures for randomly selecting research participants
• lack of random assignment of participants to treatment conditions
• poorly designed dependent measures that lack reliability and validity
• failure to account for a variety of variables related to the attitudes of students and instructors.

From design-oriented research:

• See design-based research

The problem with experimental research

“Papert (1993) sums up the inadequacy of these traditional evaluation designs: "The method of controlled experimentation that evaluates an idea by implementing it, taking care to keep everything else the same, and measuring the result, may be an appropriate way to evaluate the effects of a small modification. However, it can tell us nothing about ideas that might lead to deep change" (p. 27).” (Reeves, 1997).

Links

• Thomas C. Reeves (contributed a lot to the debates).

References

• Anderson, T. & Kanuka, H. (2002). E-Research: Issues, Strategies and Methods. Allyn Bacon.

• Robert M. Bernard, Yiping Lou, Philip C. Abrami, Lori Wozney, Evgueni Borokhovski, Peter Andrew Wallet, Anne Wade & Manon Fiset. How Does Distance Education Compare to Classroom Instruction? A Meta-analysis of the Empirical Literature. Centre for the Study of Learning and Performance, Concordia University, Montreal, Canada, and Louisiana State University. Presented as a Symposium at the Annual Meeting of The American Educational Research Association, Chicago, IL, April 2003.

• Passi, B. K. & Sudarshan Mishra, Selecting Research Areas and Research Design Approaches in Distance Education: Process Issues, The International Review of Research in Open and Distance Learning, Vol 5, No 3 (2004), ISSN: 1492-3831. HTML

• Johnson, S. D., Aragon, S. R., Shaik, N., & Palma-Rivas, N. (2000). Comparative analysis of learner satisfaction and learning outcomes in online and face-to-face learning environments. Journal of Interactive Learning Research, 11(1), 29-49.

• Reeves, T. C. (1992). Research foundations for interactive multimedia. In Promaco Conventions (Ed.), Proceedings of the International Interactive Multimedia Symposium, 177-190. Perth, Western Australia, 27-31 January. Promaco Conventions. HTML

• Reeves, T. C. (1993). Pseudoscience in computer-based instruction: The case of learner control research. Journal of Computer-Based Instruction, 20(2), 39-46.

• Reeves, Thomas, C. (1997). Evaluating What Really Matters in Computer-Based Education. HTML - HTML copy

• Reeves, T. C., (2000) Enhancing the Worth of Instructional Technology Research through "Design Experiments" and Other Development Research Strategies, Paper presented on April 27, 2000 at Session 41.29,

"International Perspectives on Instructional Technology Research for the 21st Century," a Symposium sponsored by SIG/Instructional Technology at the Annual Meeting of the American Educational Research Association, New Orleans, LA, USA. PDF

• Reeves, Thomas C. (1999) The Scope and Standards of the Journal of Interactive Learning Research, Journal of Interactive Learning Research (JILR) HTML, retrieved 19:43, 11 September 2006 (MEST).

• Reeves, T. C., (1999). A Research Agenda for Interactive Learning in the New Millennium, HTML

To sort out

• Berge, Z. L., and Mrozowski, S. (2001). Review of Research in Distance Education, 1990 to 1999. The American Journal of Distance Education, 15(3), 5 - 19.

• Garrison, R & Anderson, T. (2003). E-Learning in the 21st Century: A framework for research and practice. Routledge

• Kirkpatrick, D. (1979). Techniques for evaluating training programs. Training and Development Journal. 33(6), p. 78-92.

• Honey, M., Culp, K. M., & Carrigg, F. (1999). Perspectives on technology and education research: Lessons from the past and present. New York: Center for Children and Technology. HTML summary, retrieved 19:43, 11 September 2006 (MEST).

• Luppicini, Rocci (2003), Towards a Cyber-Constructivist Perspective (CCP) of Educational Design, Canadian Journal of Learning and Technology, Volume 29(1) Winter / hiver, 2003.

• Nettles, K., Dziuban C., Cioffe, D., Moskal, P., & Moskal, P. (2000). Technology and learning: The 'No Significant Difference' phenomenon: A structural analysis of research on technology enhanced instruction. Distributed Learning Impact Evaluation. Dziuban & Moskal (Eds.) Orlando: University of Central Florida.

• Perraton, Hilary (2000), The International Review of Research in Open and Distance Learning, Vol 1, No 1 (2000), ISSN: 1492-3831 HTML Abstract and PDF

• Reeves, Thomas C. (1995), Questioning the Questions of Instructional Technology Research, Instructional Technology Research Online, HTML

• Saba, F. (2000). Research in Distance Education: A Status Report. International Review of Research in Open and Distance Learning, 1(1). HTML

• Douglas H. Clements And Julie Sarama, Strip Mining for Gold: Research and Policy in Educational Technology\u2014A Response to \u201cFool\u2019s Gold\u201d. PDF

• http://www.academiccolab.org/resources/documents/DBR-ED-Tech4.pdf