Methodology tutorial - introduction

The educational technology and digital learning wiki
Jump to navigation Jump to search

<pageby nominor="false" comments="false"/>

Research Design for Educational Technologies - Introduction

This is part of the methodology tutorial

A first look at research

Research = ask a question and answer

Book-research-design-3.png

Major stages of a research project

Book-research-design-4.png

What do you need to know in order to get a project done ?

Book-research-design-5.png

The concept of “science”

Why bother ??

  • to understand how to write a research plan
  • to understand your academic partners (e.g. your thesis advisors)
  • to find out why they don’t like your initial research subject ...


Which elements define a given piece of research ?

  1. Theory of science:
    • what is knowledge ? academic knowledge ?
    • how should you reason ? deduce ? induce ? model ?
  2. The methodology :
    • should fit your research subject
    • ..... legitimated by some theory of science.
  3. The research object
    • you need to define exactly what you want to study
  4. The research goals
    • what’s the purpose of your study?
  5. Your means
    • time, money,
    • knowledge,
    • data access

An equilibrium between methods, object, goals and means:

Book-research-design-7.png

What does this figure tell ?

  • In educational technology research, there rarely is a ready solution for your problem !
  • There are suggestions (freedom to choose) as well as interdictions (things not to do)
  • In other words: You will have to come up with your own research design and its justification !

What do we mean by academic empirical research ( science )?

  1. A systematic activity
    • produced knowledge is a coherent whole
    • it (your results) should integrate with a system of knowledge
      (build upon literature and compare with literature)
  2. centred on reality
    • e.g. nature, la society, people’s behavior, people’s attitudes
    • in other words: don’t just speculate, look at things
  3. precise tools (hypothesis, theories, methods, reliable techniques etc.)
    • be aware of your "confirmation biais", test your conclusions against alternative explanations,
  4. generalization
    • contribute to theories by using (and testing) their theoretical statements
    • reuse (and criticize) their instruments (frameworks, analysis grids, etc.)
    • suggest modifications (or even new theories)

in addition:

  1. a belief in determinism,
    • phenomenons are the necessary consequence of conditions (causes).
    • In other words: randomness in explanation is only due to ignorance, complexity, etc.
  2. relativism :
    • our knowledge is not perfect ,
    • .... in particular in social sciences where man is subject and object, observer and observed and where many variables influence a phenomenon


What’s an interesting piece of research ?

Book-research-design-8.png You will have to produce something that is (somewhat) new

  • answer new questions
  • answer old questions without good answers
  • answer otherwise to questions addressed by the literature
  • provide support to answers found in literature with a new argumentation
  • apply a theory to a new types of cases (e.g. does it apply to Mauritius school system ?)

Book-research-design-9.png It produces something that provides “satisfaction”

  • to a certain community
    (you don’t write your thesis for yourself !).
  • to you !


The role of method and theory

Epistemological dimensions of research

Theories of science
  • Sets from a philosophical perspective the conditions of scientific knowledge
  • example: "you can’t prove a hypothesis" (only evidence, show that alternatives are

wrong,...)

Methodologies (also called approaches)
  • general recommendations on how you should design a research plan .
  • draws from a theory of science and suggests a set of legitimate methods.
  • example "you should draw hypothesis from theory and then test it with quantitative

research"

Methods
  • general recipes to study a given class of phenomenons
  • examples: "survey research methodology", "participatory software design"
Reasoning Methods
  • how to pass from data to theory and from theory to data ?
  • .... (influenced by theories of science and doctrine of approaches)
Techniques
  • practical tools to gather, manipulate, analyze data, manipulate concepts, etc.


The range of theories

  • big theories
    • go after complex topics (can’t fully be tested)
    • .... evolution of children’s minds, learning, society, .....
  • Theories with limited scope
    • concern more restricted domains
    • examples: usability guidelines for software, conditions under which multimedia animations are effective, conditions under which e-learning projects can be sustainably implemented, ...
  • Formal models
    • based on formal systems, e.g. mathematics, logics, rule systems, formal learning designs
    • sometimes tested with empirical data (not always, e.g. micro-economics is not).
  • Conceptual models
    • e.g. “systems analysis”, activity theory
    • .... conceptual tools that allow you to talk about a phenomenon, to look at them in a certain way
  • Hypothesis
    • Are frequently part of a theory or a formal model
    • Clear propositions that can be tested
    • e.g. "to introduce technology in schools, you need to provide a pedagogical support structure"

Everything together: components of knowledge

Book-research-design-20.png

The paradigm concept

  • Origin: Kuhn and his studies on “normal science”

Major components of a paradigm:

  1. a general and “asymptotic” research goal
    • ex: “understand how to teach (instructional design)”.
    • At this level you will find general ideas at what you should look at.
  2. Intermediate level: partial theories .
    • Par ex: to teach sustainable knowledge, one must engage students in practise and gradually introduce authentic problems that must be solved by themselves
  3. Operational level: Empirically tested theories .
    • For example: how to teach procedural programming, drive a car, solve a simple geometry problem.
  4. Each paradigm favors certain methodologies and provides you with "toolkits"

Why follow a paradigm ?

Hand-up.png your are much more productive if you can count on confirmed research methodology

Hand-up.png different researchers can work together , or at least profit from each other’s results

what happens if you don’t ??

Hand-down.png people will not understand you and therefore ignore you if you don’t use accepted methodology or problems

Hand-down.png your results are not comparable

The approach

Warning-icon.png There are in fact 2 different definitions

(1) approach = general methodology

  • a way to do it ...
  • includes a set of useful and tested methods for studying a set of phenomena e.g. the you could use the quasi-experimental design to study school reforms
  • an approach is often transdisciplinary:
    example: the quasi-experimental approach was developed in educational science but has been exported to public policy analysis and many other domains

(2) approach = paradigm:

  • for example: “activity theory approach” to say
    • you believe in a marxist activity-based scheme of looking at social phenomena
    • you may adopt Engeström’s related educational theories
    • you favor qualitative methodology
    • you are interested in change

A word on interdisciplinarity

  • combinations of approaches or paradigms

3 variants:

  1. multi-disciplinary :
    • juxtaposition on the same object of various research paradigms, each one keeping its own language
  2. Interdisciplinary:
    • confrontation and exchange of methods and/or adoption of a mix from various fields for a new field
  3. Trans-disciplinary:
    • usually a high abstraction level, e.g. systems theory

Difficulties:

Hand-down.png multi-disciplinary research is difficult to coordinate.

    • needs wide knowledge and very good communication skills to talk to people using another "language"

Hand-up.png Interdisciplinary research is easier

    • ..... because only methods and concepts that fit are taken from other fields,
    • however, concerned scientific communities may hate you for that
    • takes more time than disciplinary research (e.g. doing a "complete" educational technology thesis that involves pedagogy, psychology, sociology and ICT development takes more than doing a thesis in just one of these areas).

Types of research

Classification according to theory level

  1. Simple description:
    • forget it, it doesn’t have much academic value
      (unless it is led to prepare

further research)

  1. Classifications and categorizations : put order in concepts or data:
    • The intelligent case study (exploratory research)
    • Typologies (identify characteristics of classes of cases, e.g. uses of technology in

schools, types of teachers according to their beliefs in pedagogy, use of ICT, use of new pedagogies, etc.)

    • Ideal-types (theory-based identification of classes of cases)
    • The systems model (shows interactions between elements)
    • ....
  1. Research where theory plays important role:
    • Theory attempts generalization and demonstrates regularities .
    • Theory tries to understand or to explain or to predict .

Book-research-design-33.png

Scientific ends (modified from Marshall & Rossmann 95: 41)

Finalities

Typical questions

Approaches

Methods

exploratory

  • study of new phenomenons
  • preparation of another research

What happens in this program ?How does this organization work ?

  • case study
  • field study
  • participatory observation
  • in-depth interviews
  • information interviews

explanatory

  • explain the forces that constitute a phenomenon

Which events, behaviors, beliefs result in this phenomenon ?

  • comparative case study
  • historical study
  • field study
  • ethnography
  • (like above)
  • questionnaires
  • document analysis
  • field observations

descriptive/comprehen.

  • documentation of a phenomenon
  • comprehension

What are the events, structures, processes that constitute this phenomenon ?

  • field study
  • case study
  • ethnography
  • (like above)
  • non-intrusive measures
  • task observations

predictive

  • global predictions
  • predictions of events, behaviors etc.

What’s the result of X?How does X influence Y ?

  • experiment
  • quasi-experiment
  • statistical
  • simulation
  • questionnaires
  • quantitative content analysis
  • quantitative obs.

engineering

  • delivered product
  • delivered technical rule
  • test of a technical rule

What’s the problem ?How to build something ?Does it work ?What are its effects ?

  • designs (with user, usability studies)
  • most approaches above before and after engineering
  • application of design rules (technical rule)
  • rather qualitative
  • most methods above

Typology inspired by Järvinen (2004: 10)

Book-research-design-34.png

A simple typology at the end

Book-research-design-35.png

  • you may combine ...