Methodology tutorial - introduction

The educational technology and digital learning wiki
Jump to navigation Jump to search

<pageby nominor="false" comments="false"/> This is part of the methodology tutorial

Research Design for Educational Technologies - Introduction

Learning goals
Prerequisites
Moving on
Level and target population
Quality


Introduction - A first look at research

Research = ask a question and answer

Book-research-design-3.png

Major stages of a research project

Book-research-design-4.png

What do you need to know in order to get a project done ?

Book-research-design-5.png

The concept of “science”

Why bother ??

  • to understand how to write a research plan
  • to understand your academic partners (e.g. your thesis advisors)
  • to find out why they don’t like your initial research subject ...


Which elements define a given piece of research ?

  1. Theory of science:
    • what is knowledge ? academic knowledge ?
    • how should you reason ? deduce ? induce ? model ?
  2. The methodology :
    • should fit your research subject
    • ..... legitimated by some theory of science.
  3. The research object
    • you need to define exactly what you want to study
  4. The research goals
    • what’s the purpose of your study?
  5. Your means
    • time, money,
    • knowledge,
    • data access

An equilibrium between methods, object, goals and means:

Book-research-design-7.png

What does this figure tell ?

  • In educational technology research, there rarely is a ready solution for your problem !
  • There are suggestions (freedom to choose) as well as interdictions (things not to do)
  • In other words: You will have to come up with your own research design and its justification !

What do we mean by academic empirical research ( science )?

  1. A systematic activity
    • produced knowledge is a coherent whole
    • it (your results) should integrate with a system of knowledge
      (build upon literature and compare with literature)
  2. centred on reality
    • e.g. nature, la society, people’s behavior, people’s attitudes
    • in other words: don’t just speculate, look at things
  3. precise tools (hypothesis, theories, methods, reliable techniques etc.)
    • be aware of your "confirmation biais", test your conclusions against alternative explanations,
  4. generalization
    • contribute to theories by using (and testing) their theoretical statements
    • reuse (and criticize) their instruments (frameworks, analysis grids, etc.)
    • suggest modifications (or even new theories)

in addition:

  1. a belief in determinism,
    • phenomenons are the necessary consequence of conditions (causes).
    • In other words: randomness in explanation is only due to ignorance, complexity, etc.
  2. relativism :
    • our knowledge is not perfect ,
    • .... in particular in social sciences where man is subject and object, observer and observed and where many variables influence a phenomenon


What’s an interesting piece of research ?

Book-research-design-8.png You will have to produce something that is (somewhat) new

  • answer new questions
  • answer old questions without good answers
  • answer otherwise to questions addressed by the literature
  • provide support to answers found in literature with a new argumentation
  • apply a theory to a new types of cases (e.g. does it apply to Mauritius school system ?)

Book-research-design-9.png It produces something that provides “satisfaction”

  • to a certain community
    (you don’t write your thesis for yourself !).
  • to you !


The role of method and theory

Epistemological dimensions of research

Theories of science
  • Sets from a philosophical perspective the conditions of scientific knowledge
  • example: "you can’t prove a hypothesis" (only evidence, show that alternatives are

wrong,...)

Methodologies (also called approaches)
  • general recommendations on how you should design a research plan .
  • draws from a theory of science and suggests a set of legitimate methods.
  • example "you should draw hypothesis from theory and then test it with quantitative

research"

Methods
  • general recipes to study a given class of phenomenons
  • examples: "survey research methodology", "participatory software design"
Reasoning Methods
  • how to pass from data to theory and from theory to data ?
  • .... (influenced by theories of science and doctrine of approaches)
Techniques
  • practical tools to gather, manipulate, analyze data, manipulate concepts, etc.


The range of theories

  • big theories
    • go after complex topics (can’t fully be tested)
    • .... evolution of children’s minds, learning, society, .....
  • Theories with limited scope
    • concern more restricted domains
    • examples: usability guidelines for software, conditions under which multimedia animations are effective, conditions under which e-learning projects can be sustainably implemented, ...
  • Formal models
    • based on formal systems, e.g. mathematics, logics, rule systems, formal learning designs
    • sometimes tested with empirical data (not always, e.g. micro-economics is not).
  • Conceptual models
    • e.g. “systems analysis”, activity theory
    • .... conceptual tools that allow you to talk about a phenomenon, to look at them in a certain way
  • Hypothesis
    • Are frequently part of a theory or a formal model
    • Clear propositions that can be tested
    • e.g. "to introduce technology in schools, you need to provide a pedagogical support structure"

Everything together: components of knowledge

Book-research-design-20.png

The paradigm concept

  • Origin: Kuhn and his studies on “normal science”

Major components of a paradigm:

  1. a general and “asymptotic” research goal
    • ex: “understand how to teach (instructional design)”.
    • At this level you will find general ideas at what you should look at.
  2. Intermediate level: partial theories .
    • Par ex: to teach sustainable knowledge, one must engage students in practise and gradually introduce authentic problems that must be solved by themselves
  3. Operational level: Empirically tested theories .
    • For example: how to teach procedural programming, drive a car, solve a simple geometry problem.
  4. Each paradigm favors certain methodologies and provides you with "toolkits"

Why follow a paradigm ?

Hand-up.png your are much more productive if you can count on confirmed research methodology

Hand-up.png different researchers can work together , or at least profit from each other’s results

what happens if you don’t ??

Hand-down.png people will not understand you and therefore ignore you if you don’t use accepted methodology or problems

Hand-down.png your results are not comparable

The approach

Warning-icon.png There are in fact 2 different definitions

(1) approach = general methodology

  • a way to do it ...
  • includes a set of useful and tested methods for studying a set of phenomena e.g. the you could use the quasi-experimental design to study school reforms
  • an approach is often transdisciplinary:
    example: the quasi-experimental approach was developed in educational science but has been exported to public policy analysis and many other domains

(2) approach = paradigm:

  • for example: “activity theory approach” to say
    • you believe in a marxist activity-based scheme of looking at social phenomena
    • you may adopt Engeström’s related educational theories
    • you favor qualitative methodology
    • you are interested in change

A word on interdisciplinarity

  • combinations of approaches or paradigms

3 variants:

  1. multi-disciplinary :
    • juxtaposition on the same object of various research paradigms, each one keeping its own language
  2. Interdisciplinary:
    • confrontation and exchange of methods and/or adoption of a mix from various fields for a new field
  3. Trans-disciplinary:
    • usually a high abstraction level, e.g. systems theory

Difficulties:

Hand-down.png multi-disciplinary research is difficult to coordinate.

  • needs wide knowledge and very good communication skills to talk to people using another "language"


Hand-up.png Interdisciplinary research is easier

  • ..... because only methods and concepts that fit are taken from other fields,
  • however, concerned scientific communities may hate you for that
  • takes more time than disciplinary research (e.g. doing a "complete" educational technology thesis that involves pedagogy, psychology, sociology and ICT development takes more than doing a thesis in just one of these areas).


Types of research

Classification according to theory level

  1. Simple description:
    • forget it, it doesn’t have much academic value (unless it is led to prepare further research)
  2. Classifications and categorizations : put order in concepts or data:
    • The intelligent case study (exploratory research)
    • Typologies (identify characteristics of classes of cases, e.g. uses of technology in schools, types of teachers according to their beliefs in pedagogy, use of ICT, use of new pedagogies, etc.)
    • Ideal-types (theory-based identification of classes of cases)
    • The systems model (shows interactions between elements)
    • ....
  3. Research where theory plays important role:
    • Theory attempts generalization and demonstrates regularities .
    • Theory tries to understand or to explain or to predict .

Hand-right.png Research should aspire to level 3 !

Scientific ends

Modified from Marshall & Rossmann (1995: 41)

Finalities

Typical questions

Approaches

Methods

exploratory

  • study of new phenomenons
  • preparation of another research

What happens in this program ?How does this organization work ?

  • case study
  • field study
  • participatory observation
  • in-depth interviews
  • information interviews

explanatory

  • explain the forces that constitute a phenomenon

Which events, behaviors, beliefs result in this phenomenon ?

  • comparative case study
  • historical study
  • field study
  • ethnography
  • (like above)
  • questionnaires
  • document analysis
  • field observations

descriptive/comprehen.

  • documentation of a phenomenon
  • comprehension

What are the events, structures, processes that constitute this phenomenon ?

  • field study
  • case study
  • ethnography
  • (like above)
  • non-intrusive measures
  • task observations

predictive

  • global predictions
  • predictions of events, behaviors etc.

What’s the result of X?How does X influence Y ?

  • experiment
  • quasi-experiment
  • statistical
  • simulation
  • questionnaires
  • quantitative content analysis
  • quantitative obs.

engineering

  • delivered product
  • delivered technical rule
  • test of a technical rule

What’s the problem ?How to build something ?Does it work ?What are its effects ?

  • designs (with user, usability studies)
  • most approaches above before and after engineering
  • application of design rules (technical rule)
  • rather qualitative
  • most methods above

Typology inspired by Järvinen (2004: 10)

Book-research-design-34.png

This typology is interesting because it integrates engineering sciences (something social science methdology texts usually do not. Since part of educational technology is research on technologies, the Järvinen typology is useful.

A simple typology at the end

In further installements of the methodology tutorials we will focus on these broad categories of research.

Book-research-design-35.png

You may combine of course elements from several research types...

References

  • Järvinnen, (2004), On research methods, Tampere: Opinpajan Kirja. [Methods book for ICT-related issues, good but difficult and dense reading]
  • Marshall, C. & Rossman, G. B. (1995) , Designing Qualitative Research, second edition, Sage, London. [conseillé aux débutants]

See Research methodology resources for web links and a larger bibliography.