by Martina Kelly, MA, MBBCh | Helen Reid, MD | Deirdre Bennett MB, MSc, MA, PhD | Sarah Yardley, MD, PhD | Tim Dornan, MD, MSc, PhD
The recent burgeoning of qualitative evidence synthesis (QES) methodologies has widened opportunities for health researchers to review complex questions. These approaches are particularly suited to multifaceted fields such as humanism. Unfortunately, increasing QES activity has been accompanied by a plethora of terminology, and deciding which methodology to use can be confusing.
This introduction to QES will provide a definition, consider some key principles of systematic review, describe some commonly used methods and link to exemplar papers for interested readers.
What is QES?
Bearman and Dawson define qualitative evidence synthesis as “any methodology whereby study findings are systematically interpreted through a series of expert judgments to represent the meaning of the collected work. In (QES), the findings of qualitative studies – and sometimes mixed-methods and quantitative research – are pooled.”
Two important principles in systematic review are
- Meta — higher, beyond
- Synthesis — combining components to form a connected whole where the sum is greater than its parts
Searching & Sampling in QES
In conventional reviews, searching is typically pre-determined by specific inclusion and exclusion criteria. This usually precedes data extraction. In qualitative reviews, particularly those using an interpretative approach, it is common for searching and data extraction to take place an an iterative manner. Qualitative approaches to sampling such as theoretical sampling or purposive sampling may be used. (Booth 2016)
Quality appraisal in QES
Quality appraisal is a contentious issue in qualitative systematic reviews: some argue that it stifles creativity and detracts from the insights the diverse approaches of qualitative research afford. More recently however the argument seems to be shifting from if to how, quality appraisal should be performed and careful consideration needs to be given to criteria used. (Booth 2016 and Hannes 2011)
Different types of QES vary in their approach to sampling and quality appraisal. What’s important is that authors’ decisions are made explicit and justified, and are consistent with the worldview in which the review is performed.(Gordon 2016)
Types of QES
Meta-ethnography is the most common form of QES in health research, likely because it was the first. The term may lead to confusion; to clarify — all types of qualitative study can be included in a meta-ethnography — not just ethnographic research.
Noblit and Hare’s work was seminal because they presented for the first time a way to synthesize qualitative studies. A key element of meta-ethnographic approaches is to use comparative understanding rather than aggregating data. Three major strategies may be adopted:
- Reciprocal translation: Authors identify key themes, metaphors or concepts and attempt to translate these into each other
- Fefutational synthesis: Authors characterize and interpret contradictions
- Line of argument synthesis: Authors build a general interpretation
Often the output of a meta-ethnography is a theory which has greater explanatory power than the individual studies.
- Although this paper is dated, it is helpful as the authors work through the seven steps suggested by Noblit & Hare — Britten N, Campbell R, Pope C, Donovan J, Morgan M, Pill R. Using meta ethnography to synthesise qualitative research: A worked example. Journal of Health Services Research & Policy. 2002;7(4):209-15.
- This paper provides an insider view of what its like to try a method; it highlights how messy these approaches can be and how to work through them — Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J. Conducting a meta-ethnography of qualitative literature: Lessons learnt. BMC Medical Research Methodology. 2008;8(1):21.
Realist review or realist synthesis aims to describe how complex interventions work in different contexts. A complex intervention refers to one whose outcomes depend on the interaction between participants and the opportunities or resources provided to them. Realist review provides an interpretive, theory-driven narrative summary of the literature (quantitative and/or qualitative). Traditional systematic reviews of complex interventions often produce mixed findings with no information as to why something worked or how. Realist review addresses this shortcoming. It does not provide simple answers to complex questions, but provides rich context specific information which is useful to policy makers and practitioners.
- Wong G, Greenhalgh T, Westhorp G, Pawson R. Realist methods in medical education research: what are they and what can they contribute? Medical Education. 2012;46(1):89-96.
- Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: Realist syntheses. BMC Medicine. 2013;11(1):1
Meta-narrative review or meta-narrative synthesis (the terms are used interchangeably) aims to highlight what different schools or traditions of literature might learn from one another’s approaches. It was developed as a practical response to QES challenges faced when teams from very different research traditions approach the same topic of study. In their seminal paper on the diffusion of innovations Greenhalgh et al encountered literature from research traditions as diverse as rural sociology, clinical epidemiology and marketing. Their findings (and the standards by which each tradition might appraise the “quality” of evidence) differed hugely. By consciously and reflexively stepping out of their own world view, Greenhalgh’s team was able to summarize the different perspectives on the same topic in an over-arching narrative.
- Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. Storylines of research in diffusion of innovation: A meta-narrative approach to systematic review. Social Science & Medicine. 2005;61(2):417-30.
- Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: Meta-narrative reviews. BMC Medicine 2013; 11:20.
Widely used in health research and combines elements of grounded theory and meta-ethnography. The central process involves identifying common themes across the literature and summarizing the findings accordingly.
- Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology. 2008;8(1):1.
Meta-study approaches a topic by recognizing that method and theory are inextricably linked to findings and considers ways in which method and theory shape research. Thus the process of synthesis extends beyond results (meta-analysis) to look at method (meta-method) and theory (meta-theory) as well. These are then integrated iteratively through a process of interpreting, theorizing and reflecting to create a comprehensive understanding of a topic or field of research (meta-synthesis).
- The seminal paper on meta-study — Thorne S, Paterson B, Acorn S, Canam C, Joachim G, Jillings C. Chronic illness experience: Insights from a metastudy. Qualitative Health Research. 2002;12(4):437-52.
- This paper makes the approach seem easier to understand and is a nice example relevant to humanism — Edwards A, Pang N, Shiu V, Chan C. The understanding of spirituality and the potential role of spiritual care in end-of-life and palliative care: A meta-study of qualitative research. Palliative Medicine. 2010. But this one
Critical Interpretative synthesis
Critical interpretative synthesis is an interpretative form of QES which aims to develop concepts and theoretical understandings with explanatory power. Reviewers adopt a critical approach to the decisions made by primary authors. This involves includes questioning how a topic has been problematized, underlying assumptions and factors influencing proposed solutions/results, including how a study is reported. Within this framework the research question is iteratively refined and sampling is more likely to be purposive and/ or theoretical. It demands much reflexivity on behalf of the study team.
- Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, et al. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Medical Research Methodology. 2006;6(1):35.
Scoping studies take a broad approach to map a research topic in order to examine the main sources and types of evidence available. Scoping reviews are conducted to examine the extent, range and nature of research activity, summarize the findings from existing research and identify gaps in the existing literature. Although the term “rapid” is used in the original definition, scoping reviews can be arduous as identifying some types of literature e.g. grey literature, can be challenging. Arksey & O’Malley’s initial outline is further elaborated by Levac. Quality appraisal is an optional element in scoping studies. A 6th step, the consultation exercise, can act as a knowledge translation component to the review, and although advocated it is not often reported.
Although scoping studies are not QES per se, given the diverse nature of data included, authors often use narrative methods to present findings. As such, findings tend to be more aggregative than interpretative, but this can vary widely according to the research question.
- Arksey H, O’Malley L. Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology. 2005;8(1):19-32.
- Levac D, Colquhoun H, O’Brien KK. Scoping studies: Advancing the methodology. Implement Science. 2010;5(1):1-9.
- Tricco AC, Lillie E, Zarin W, O’Brien K, Colquhoun H, Kastner M, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Medical Research Methodology. 2016;16(1):1.
Starting Out in QES: Some Reflections
As someone medically trained and familiar with traditional systematic review, engaging with QES methodologies can be intimidating. For example, the idea of refining the research question whilst iteratively searching the literature can feel disorderly.
Quality appraisal of studies is challenging, given the wide variety and heterogeneity of studies that may be included. How does one account for a study, which may be methodologically flawed but contests or reveals unique insights about the phenomena under scrutiny?
Working in a multidisciplinary team, with colleagues experienced in qualitative research is essential. The principles of qualitative research remain constant across the methodologies, the nuances are fun to learn.
Being reflexive can be uncomfortable.
Synthesis is messy, time consuming and fuzzy – from all of this somehow clarity will materialize (having faith in that is very uncomfortable!).
BUT the process is intellectually stimulating and the product, hopefully, has potential to make sense of ambiguous but central research topics in humanism.
Interested in learning more about QES?
- Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: A critical review. BMC Medical Research Methodology. 2009;9(1):59.
- Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Systematic Reviews. 2012;1(1):1.
- This paper has a nice chart comparing methods — Kastner M, Tricco ACm Soobiah C, Lille E, Perrier L, Horsely T, Welch V, Cogo E, Antony J, Straus S. What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Medical Research Methodology. 2012. 12:114.
- Although this is an older article, it is written by key leaders in the field of QES — Thorne S, Jensen L, Kearney MH, Noblit G, Sandelowski M. Qualitative metasynthesis: Reflections on methodological orientation and ideological agenda. Qualitative Health Research. 2004;14(10):1342-65.–