Methodology tutorial - introduction

<pageby nominor="false" comments="false"/>

Research Design for Educational Technologies - Introduction

This is part of the methodology tutorial

A first look at research

Research = ask a question and answer

Major stages of a research project

What do you need to know ?

Objectives of these tutorials

The concept of “science”

Why bother ??

• to understand how to write a research plan
• to understand your academic partners (e.g. your thesis advisors)
• to find out why they don’t like your initial research subject ...

Which elements define a given piece of research ?

1. Theory of science:
   • what is knowledge ? academic knowledge ?
   • how should you reason ? deduce ? induce ? model ?
2. The methodology :
   • should fit your research subject
   • ..... legitimated by some theory of science.
3. The research object
   • you need to define exactly what you want to study
4. The research goals
   • what’s the purpose of your study?
5. Your means
   • time, money,
   • knowledge,
   • data access

An equilibrium between methods, object, goals and means:

What does this figure tell ?

• In educational technology research, there rarely is a ready solution for your problem !
• There are suggestions (freedom to choose) as well as interdictions (things not to do)
• In other words: You will have to come up with your own research design and its justification !

What do we mean by academic empirical research ( science )?

1. A systematic activity
   • produced knowledge is a coherent whole
   • it (your results) should integrate with a system of knowledge
     (build upon literature and compare with literature)
2. centred on reality
   • e.g. nature, la society, people’s behavior, people’s attitudes
   • in other words: don’t just speculate, look at things
3. precise tools (hypothesis, theories, methods, reliable techniques etc.)
   • be aware of your "confirmation biais", test your conclusions against alternative explanations,
4. generalization
   • contribute to theories by using (and testing) their theoretical statements
   • reuse (and criticize) their instruments (frameworks, analysis grids, etc.)
   • suggest modifications (or even new theories)

in addition:

1. a belief in determinism,
   • phenomenons are the necessary consequence of conditions (causes).
   • In other words: randomness in explanation is only due to ignorance, complexity, etc.
2. relativism :
   • our knowledge is not perfect ,
   • .... in particular in social sciences where man is subject and object, observer and observed and where many variables influence a phenomenon

What’s an interesting piece of research ?

You will have to produce something that is (somewhat) new

• answer new questions
• answer old questions without good answers
• answer otherwise to questions addressed by the literature
• provide support to answers found in literature with a new argumentation
• apply a theory to a new types of cases (e.g. does it apply to Mauritius school system ?)

It produces something that provides “satisfaction”

• to a certain community
  (you don’t write your thesis for yourself !).
• to you !

The role of method and theory

Epistemological dimensions of research

Theories of science

• Sets from a philosophical perspective the conditions of scientific knowledge
• example: "you can’t prove a hypothesis" (only evidence, show that alternatives are

wrong,...)

Methodologies (also called approaches)

• general recommendations on how you should design a research plan .
• draws from a theory of science and suggests a set of legitimate methods.
• example "you should draw hypothesis from theory and then test it with quantitative

research"

Methods

• general recipes to study a given class of phenomenons
• examples: "survey research methodology", "participatory software design"

Reasoning Methods

• how to pass from data to theory and from theory to data ?
• .... (influenced by theories of science and doctrine of approaches)

Techniques

• practical tools to gather, manipulate, analyze data, manipulate concepts, etc.

The range of theories

• big theories
  • go after complex topics (can’t fully be tested)
  • .... evolution of children’s minds, learning, society, .....

• Theories with limited scope
  • concern more restricted domains
  • examples: usability guidelines for software, conditions under which multimedia animations are effective, conditions under which e-learning projects can be sustainably implemented, ...

• Formal models
  • based on formal systems, e.g. mathematics, logics, rule systems, formal learning designs
  • sometimes tested with empirical data (not always, e.g. micro-economics is not).

• Conceptual models
  • e.g. “systems analysis”, activity theory
  • .... conceptual tools that allow you to talk about a phenomenon, to look at them in a certain way

• Hypothesis
  • Are frequently part of a theory or a formal model
  • Clear propositions that can be tested
  • e.g. "to introduce technology in schools, you need to provide a pedagogical support structure"

Everything together: components of knowledge

The paradigm concept

• Origin: Kuhn and his studies on “normal science”

Major components of a paradigm:

1. a general and “asymptotic” research goal
   • ex: “understand how to teach (instructional design)”.
   • At this level you will find general ideas at what you should look at.
2. Intermediate level: partial theories .
   • Par ex: to teach sustainable knowledge, one must engage students in practise and gradually introduce authentic problems that must be solved by themselves
3. Operational level: Empirically tested theories .
   • For example: how to teach procedural programming, drive a car, solve a simple geometry problem.
4. Each paradigm favors certain methodologies and provides you with "toolkits"

Why follow a paradigm ?

your are much more productive if you can count on confirmed research methodology

different researchers can work together , or at least profit from each other’s results

what happens if you don’t ??

people will not understand you and therefore ignore you if you don’t use accepted methodology or problems

your results are not comparable

The approach

There are in fact 2 different definitions

(1) approach = general methodology

• a way to do it ...
• includes a set of useful and tested methods for studying a set of phenomena e.g. the you could use the quasi-experimental design to study school reforms
• an approach is often transdisciplinary:
  example: the quasi-experimental approach was developed in educational science but has been exported to public policy analysis and many other domains

(2) approach = paradigm:

• for example: “activity theory approach” to say
  • you believe in a marxist activity-based scheme of looking at social phenomena
  • you may adopt Engeström’s related educational theories
  • you favor qualitative methodology
  • you are interested in change

A word on interdisciplinarity

• combinations of approaches or paradigms

3 variants:

1. multi-disciplinary :
   • juxtaposition on the same object of various research paradigms, each one keeping its own language
2. Interdisciplinary:
   • confrontation and exchange of methods and/or adoption of a mix from various fields for a new field
3. Trans-disciplinary:
   • usually a high abstraction level, e.g. systems theory

Difficulties:

multi-disciplinary research is difficult to coordinate.

• • needs wide knowledge and very good communication skills to talk to people using another "language"

Interdisciplinary research is easier

• • ..... because only methods and concepts that fit are taken from other fields,
  • however, concerned scientific communities may hate you for that
  • takes more time than disciplinary research (e.g. doing a "complete" educational technology thesis that involves pedagogy, psychology, sociology and ICT development takes more than doing a thesis in just one of these areas).

Types of research

Classification according to theory level

1. Simple description:
   • forget it, it doesn’t have much academic value
     (unless it is led to prepare

further research)

1. Classifications and categorizations : put order in concepts or data:
   • The intelligent case study (exploratory research)
   • Typologies (identify characteristics of classes of cases, e.g. uses of technology in

schools, types of teachers according to their beliefs in pedagogy, use of ICT, use of new pedagogies, etc.)

1. • Ideal-types (theory-based identification of classes of cases)
   • The systems model (shows interactions between elements)
   • ....
2. Research where theory plays important role:
   • Theory attempts generalization and demonstrates regularities .
   • Theory tries to understand or to explain or to predict .

Scientific ends (modified from Marshall & Rossmann 95: 41)

Finalities	Typical questions	Approaches	Methods
exploratory study of new phenomenons preparation of another research	What happens in this program ?How does this organization work ?	case study field study	participatory observation in-depth interviews information interviews
explanatory explain the forces that constitute a phenomenon	Which events, behaviors, beliefs result in this phenomenon ?	comparative case study historical study field study ethnography	(like above) questionnaires document analysis field observations
descriptive/comprehen. documentation of a phenomenon comprehension	What are the events, structures, processes that constitute this phenomenon ?	field study case study ethnography	(like above) non-intrusive measures task observations
predictive global predictions predictions of events, behaviors etc.	What’s the result of X?How does X influence Y ?	experiment quasi-experiment statistical simulation	questionnaires quantitative content analysis quantitative obs.
engineering delivered product delivered technical rule test of a technical rule	What’s the problem ?How to build something ?Does it work ?What are its effects ?	designs (with user, usability studies) most approaches above before and after engineering	application of design rules (technical rule) rather qualitative most methods above

Typology inspired by Järvinen (2004: 10)

A simple typology at the end

• you may combine ...