Mixed methods

Draft

Introduction

Mixed methods usually refer to some kind of Triangulation: “The combinations and comparisons of multiple data sources, data collection and analysis procedures, research methods, or inferences that occur at the end of a study., Denzin (1978) used the terms data triangulation, theory triangulation and methodological triangulation. Erzberger and Udo have used the term to refer to agreement between inferences.” (Glossary of Mixed Methods Terms/Concepts, retrieved 19:19, 27 January 2012 (CET).

Mixed methods researchers often have a societal agenda. For example: “Mixed methods research provides an antidualistic and syncretic philosophy and set of approaches or possibilities for merging insights from diverse perspectives; its working goal is to provide pragmatic, ethical solutions to local and societal problems.” (Johnson, 2009).

The rationale for mixed methods

Mixed methods ....

Fortunately, many (or most?) qualitative researchers and quantitative researchers (i.e., postpositivists) have now reached basic agreement on several points of earlier philosophical disagreement (e.g., Phillips & Burbules, 2000; Reichardt & Cook, 1979; Reichardt & Rallis, 1994). Basic agreement has been reached on each of the following issues: (a) the relativity of the “light of reason” (i.e., what appears reasonable can vary across persons); (b) theory-laden perception or the theory-ladenness of facts (i.e., what we notice and observe is affected by our background knowledge, theories, and experiences; in short, observation is not a perfect and direct window into “reality”); (c) underdetermination of theory by evidence (i.e., it is possible for more than one theory to fit a single set of empirical data); (d) the Duhem-Quine thesis or idea of auxiliary assumptions (i.e., a hypothesis cannot be fully tested in isolation because to make the test we also must make various assumptions; the hypothesis is embedded in a holistic network of beliefs; and alternative explanations will continue to exist); (e) the problem of induction (i.e., the recognition that we only obtain probabilistic evidence, not final proof in empirical research; in short, we agree that the future may not resemble the past); (f) the social nature of the research enterprise (i.e., researchers are embedded in communities and they clearly have and are affected by their attitudes, values, and beliefs); and (g) the value-ladenness of inquiry (this is similar to the last point but specifically points out that human beings can never be completely value free, and that values affect what we choose to investigate, what we see, and how we interpret what we see).

(Johnson & Onwuegbuzie, 2004: 16)

Varieties of mixed designs

The following variants were found in the Glossary of Mixed Methods Terms/Concepts (retrieved 19:19, 27 January 2012 (CET)). The glossary summarizes terms were adopted from Tashakkori and Teddlie's (2003) Handbook of mixed methods in the social and behavioral research.

Notice with respect to copyright: Since the glossary was last updated in 2007 and the front page on 2009 and since there are broken pages, we fear that this valuable resource could disappear. We therefore decided to reproduce some valuable definitions here - Daniel K. Schneider 19:19, 27 January 2012 (CET).

• Concurrent Mixed Method Design: This is a multistrand design in which both QUAL and QUAN data are collected and analyzed to answer a single type of research question (either QUAL or QUAN). The final inferences are based on both data analysis results. The two types of data are collected independently at the same time or with a time lag.

• Concurrent Mixed Model Design: This is a multistrand mixed design in which there are two relatively independent strands/phases: one with QUAL questions and data collection and analysis techniques and the other with QUAN questions and data collection and analysis techniques. The inferences made on the basis of the results of each strand are pulled together to form meta-inferences at the end of the study. See also rules of integration.

• Concurrent Nested Design: This is a concurrent mixed model design classified on the basis of (conceptual or paradigmatic) dominance or priority of the study. In this design, a quantitative strand/phase is embedded within a predominantly qualitative study (quan + QUAL) or vice versa (QUAN + qual). QUAL and QUAN approaches are used to “confirm, cross-validate, or corroborate findings within a single study” (Creswell, Plano Clark, Gutmann, & Hanson, 2003).

• Concurrent Triangulation Design: This is a concurrent mixed model design classified on the basis of purpose of the study. In this design, QUAL and QUAN approaches are used to “confirm, cross-validate, or corroborate findings within a single study” (Creswell et al., 2003).

• Conversion Mixed Model Design: This is a multistrand concurrent design in which mixing of QUAL and QUAN approaches occurs in all components/stages, with data transformed (qualitized or quantitized) and analyzed both qualitatively and quantitatively.

• Fully integrated mixed model design: This is a multistrand concurrent design in which mixing of QUAL and QUAN approaches occurs in an interactive (i.e., dynamic, reciprocal, interdependent, iterative) manner at all stages of the study. At each stage (e.g., in formulating questions), one approach (e.g., QUAL) affects the formulation of the other (e.g., QUAN).

• Mixed Method Design: (1) This is a design that includes both QUAL and QUAN data collection and analysis in parallel form (concurrent mixed method design, in which two types of data are collected and analyzed), in sequential form (sequential mixed method design, in which one type of data provides a basis for collection of another type of data), or where the data are converted (qualitized or quantitized) and analyzed again (conversion mixed method design). (2) (Bazeley, 2003) This design includes studies that “use mixed data (numerical and text) and alternative tools (statistics and text analysis) but apply the same method, for example, in developing a grounded theory.”

• Mixed model design: This is a design in which mixing of QUAL and QUAN approaches occurs in all stages of the study (formulation of research questions, data collection procedures and research method, and interpretation of the results to make final inferences) or across stages of the study (e.g., QUAL questions, QUAN data). In multistrand designs, either the strands are parallel (concurrent mixed model design) or sequential (sequential mixed model design, in which inferences of one strand lead to questions of the next strand) or the data are converted and analyzed again to answer different questions (conversion mixed model design).

• Multilevel mixed methods design: This is a design in which QUAL data are collected at one level (e.g., child), and QUAN data are collected at another level (e.g., family) in a concurrent or sequential manner to answer different aspects of the same research question. Both types of data are analyzed accordingly, and the results are used to make inferences. Because the questions and inferences all are in one approach (QUAL or QUAN), this is a predominantly QUAL or QUAN study with some added components. In practice, because research questions and the inferences that are made at the end of the study are usually both QUAL and QUAN (using mixed models), this design is not common.

• Multilevel mixed model design: This is a design in which QUAL data are collected at one level (e.g., child) and QUAN data are collected at another level (e.g., family) in a concurrent or sequential manner to answer interrelated research questions with multiple approaches (QUAL and QUAN). Both types of data are analyzed accordingly, and the results are used to make multiple types of inferences (QUAL and QUAN) that are pulled together at the end of the study in the form of “global inferences.” See also multilevel mixed method design.

• Multimethods design: This refers to designs in which the research questions are answered by using two data collection procedures or two research methods, both with either the QUAL or QUAN approach. See also multimethods QUAL study and multimethods QUAN study.

• Multimethods QUAL study: This refers to designs in which the research questions are answered by using two QUAL data collection procedures or two QUAL research methods.

• Multimethods QUAN study: This refers to designs in which the research questions are answered by using two QUAN data collection procedures or two QUAN research methods.

• Multiple methods design: (Brewer & Hunter, 2003) This refers to designs in which more than one research method or data collection and analysis technique is used to answer research questions. They include mixed methods designs (QUAL + QUAN) and multimethods designs (QUAN + QUAN or QUAL + QUAL).

• Multistrands design: This refers to designs that use more than one research method or data collection procedure. See also multimethods design.

• Parallel mixed model design: See concurrent mixed model design.

• Sequential explanatory design: According to Creswell et al.(2003), this design “is characterized by the collection and analysis of quantitative data followed by the collection and analysis of qualitative data. Priority is typically given to the quantitative data, and the two methods are integrated during the interpretation phase of the study.”

• Sequential exploratory design: According to Creswell, et al. (2003), this design “is characterized by an initial phase of qualitative data collection and analysis, followed by a phase of quantitative data collection and analysis. Therefore, the priority is given to the qualitative aspects of the study.

• Sequential Mixed Method Design: (Onwuegbuzie and Teddlie, Chapter 13, this volume.) A design in which one type of data (e.g. QUAN) provides a basis for the collection of another type of data (e.g. QUAL). It answers one type of question (QUAL or QUAN) by collecting and analyzing two types of data (QUAL and QUAN). Inferences are based on the analysis of both types of data. This term subsumes “sequential study, “two‑phase design ,” “sequential QUAL-QUAN Analysis” and “sequential QUAN-QUAL analysis”.

• Sequential Mixed Model Design: A multi-strand mixed (QUAL-QUAN, or QUAN-QUAL) design in which the conclusions that are made on the basis of the results of the first strand (e.g. a QUAN phase) lead to formulation of questions, data collection, and data analysis for the next strand (e.g. a QUAL phase). The final inferences are based on the results of both strands of the study. The second strand/phase of the study is conducted to either confirm/disconfirm the inferences of the first strand, or to provide further explanation for unexpected findings of the first strand.

• Transformative mixed methods design: (Creswell et al., Chapter 8, this volume) This refers to a research project that Creswell, et al. describe as follows: “In both perspective and outcomes, it is dedicated to promoting change at levels ranging from the personal to the political. Furthermore, it is possible to conduct any quantitative, qualitative, or mixed methods study with a transformative or advocacy purpose.”

• Two Phase Design:. (Currall & Towler, Chapter 18, this volume) A study with a qualitative phase followed by a quantitative phase or vice-versa. See Multistrand Design.

See also:

• Methodology tutorial
• Research methodology resources

Software

See: Research methodology resources and Computer assisted qualitative research analysis software

Online services:

• DeDoose (not tested)

Links

• Multimethodology (Wikipedia).

• Mixed Methods Network for Behavioral, Social, and Health Sciences (includes concepts, courses, publications, links, etc.)

• LinkedIn Bridges group

• Mixed-Methods Special Interest Group of AERA (old page ?)

Bibliography

• Brannen, Julia. 2005. “Mixing Methods: The Entry of Qualitative and Quantitative Approaches into the Research Process.” International Journal of Social Research Methodology 8:173-184.

• Brewer, J., & Hunter, A. (1989). Multimethod research: A synthesis of style. Newbury Park, CA: Sage.

• Creswell JW, Plano Clark VL (2007). Designing and conducting mixed methods research, Thousand Oaks: Sage.

• Creswell, John W. (2002). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (2nd Edition), Sage Publications, ISBN 0761924426

• Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11, 255-274.

• Greene, JC (2007) Mixed methods in social inquiry. San Francisco: Jossey-Bass.

• Howe, K. R. (1988). Against the quantitative-qualitative incompatibility thesis or dogmas die hard. Educational Researcher, 17, 10-16.

• Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26. Abstract/PDF (Access restricted)

• Johnson R.B., Onwuegbuzie A.J., Turner L.A. (2007). Toward a Definition of Mixed Methods Research, Journal of Mixed Methods Research, April 2007 vol. 1 no. 2 112-133. Abstract/PDF

• Johnson R.B. Comments on Howe: Toward a more inclusive "scientific research in education", (2009) Educational Researcher, 38 (6), pp. 449-457. Abstract.

• Johnson RB, Christensen, LB (2008). Educational research: Quantitative, qualitative, and mixed approaches, 3rd ed., Thousand Oaks, CA: Sage.

• Lowenthal, P. R., & Leech, N. (2009). Mixed research and online learning: Strategies for improvement. In T. T. Kidd (Ed.), Online education and adult learning: New frontiers for teaching practices (pp. 202–211). Hershey, PA: IGI Global.

• Tashakkori, A. & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage.

• Tashakkori, A. & Teddlie, C. (2003). Handbook of mixed methods in the social and behavioral research. Thousand Oaks, CA: Sage.

• Tashakkori, A. & Teddlie, C. (2003b). Issues and dilemmas in teaching research methods courses in social and behavioral sciences: A U.S. perspective. International Journal of Social Research Methodology, 6 (1), 61 - 77.

• Teddlie C, Johnson R.B. (2009a), "Foundations of mixed methods research: Integrating quantitative and qualitative techniques in the social and behavioral sciences, Methodological thought before the 20th century", in Teddlie C, Tashakkori A. (eds). Foundations of mixed methods research: Integrating quantitative and qualitative techniques in the social and behavioral sciences, Sage 40–61.

• Teddlie C & Tashakkori A (2009). Foundations of mixed methods research: Integrating quantitative and qualitative techniques in the social and behavioral sciences. Thousand Oaks: Sage.