The Research Institute

Introduction to Qualitative Evidence Synthesis

by Martina Kelly, MA, MBBCh | Helen Reid, MD | Deirdre Bennett MB, MSc, MA, PhD | Sarah Yardley, MD, PhD | Tim Dornan, MD, MSc, PhD

The recent burgeoning of qualitative evidence synthesis (QES) methodologies has widened opportunities for health researchers to review complex questions.  These approaches are particularly suited to multifaceted fields such as humanism. Unfortunately, increasing QES activity has been accompanied by a plethora of terminology, and deciding which methodology to use can be confusing.

This introduction to QES will provide a definition, consider some key principles of systematic review, describe some commonly used methods and link to exemplar papers for interested readers.

What is QES?

Bearman and Dawson define qualitative evidence synthesis as “any methodology whereby study findings are systematically interpreted through a series of expert judgments to represent the meaning of the collected work. In (QES), the findings of qualitative studies – and sometimes mixed-methods and quantitative research – are pooled.”

Two important principles in systematic review are

  • Meta — higher, beyond
  • Synthesis — combining components to form a connected whole where the sum is greater than its parts
Searching & Sampling in QES

In conventional reviews, searching is typically pre-determined by specific inclusion and exclusion criteria.  This usually precedes data extraction. In qualitative reviews, particularly those using an interpretative approach, it is common for searching and data extraction to take place an an iterative manner. Qualitative approaches to sampling such as theoretical sampling or purposive sampling may be used.  (Booth 2016)

Quality appraisal in QES

Quality appraisal is a contentious issue in qualitative systematic reviews: some argue that it stifles creativity and detracts from the insights the diverse approaches of qualitative research afford.  More recently however the argument seems to be shifting from if to how, quality appraisal should be performed and careful consideration needs to be given to criteria used. (Booth 2016 and Hannes 2011)

Different types of QES vary in their approach to sampling and quality appraisal.  What’s important is that authors’ decisions are made explicit and justified, and are consistent with the worldview in which the review is performed.(Gordon 2016)

Types of QES

Meta-ethnography

Meta-ethnography is the most common form of QES in health research, likely because it was the first.  The term may lead to confusion; to clarify — all types of qualitative study can be included in a meta-ethnography — not just ethnographic research.

Noblit and Hare’s  work was seminal because they presented for the first time a way to synthesize qualitative studies. A key element of meta-ethnographic approaches is to use comparative understanding rather than aggregating data. Three major strategies may be adopted:

  • Reciprocal translation: Authors identify key themes, metaphors or concepts and attempt to translate these into each other
  • Fefutational synthesis: Authors characterize and interpret contradictions
  • Line of argument synthesis: Authors build a general interpretation

Often the output of a meta-ethnography is a theory which has greater explanatory power than the individual studies.

More information:

Realist Review

Realist review or realist synthesis aims to describe how complex interventions work in different contexts. A complex intervention refers to one whose outcomes depend on the interaction between participants and the opportunities or resources provided to them.  Realist review provides an interpretive, theory-driven narrative summary of the literature (quantitative and/or qualitative). Traditional systematic reviews of complex interventions often produce mixed findings with no information as to why something worked or how. Realist review addresses this shortcoming. It does not provide simple answers to complex questions, but provides rich context specific information which is useful to policy makers and practitioners.

More information:

Meta-narrative review

Meta-narrative review or meta-narrative synthesis (the terms are used interchangeably) aims to highlight what different schools or traditions of literature might learn from one another’s approaches. It was developed as a practical response to QES challenges faced when teams from very different research traditions approach the same topic of study.  In their seminal paper on the diffusion of innovations Greenhalgh et al encountered literature from research traditions as diverse as rural sociology, clinical epidemiology and marketing.  Their findings (and the standards by which each tradition might appraise the “quality” of evidence) differed hugely.  By consciously and reflexively stepping out of their own world view, Greenhalgh’s team was able to summarize the different perspectives on the same topic in an over-arching narrative.

More information:

Thematic synthesis

Widely used in health research and combines elements of grounded theory and meta-ethnography. The central process involves identifying common themes across the literature and summarizing the findings accordingly.

More information:

Meta-study

Meta-study approaches a topic by recognizing that method and theory are inextricably linked to findings and considers ways in which method and theory shape research.  Thus the process of synthesis extends beyond results (meta-analysis) to look at method (meta-method) and theory (meta-theory) as well. These are then integrated  iteratively through a process of interpreting, theorizing and reflecting to create a comprehensive understanding of a topic or field of research (meta-synthesis).

More information:

Critical Interpretative synthesis

Critical interpretative synthesis is an interpretative form of QES which aims to develop concepts and theoretical understandings with explanatory power.  Reviewers adopt a critical approach to the decisions made by primary authors. This involves includes questioning how a topic has been problematized, underlying assumptions and factors influencing proposed solutions/results, including how a study is reported. Within this framework the research question is iteratively refined and sampling is more likely to be purposive and/ or theoretical. It demands much reflexivity on behalf of the study team.

More information:

Scoping study

Scoping studies take a broad approach to map a research topic in order to examine the main sources and types of evidence available. Scoping reviews are conducted to examine the extent, range and nature of research activity, summarize the findings from existing research and identify gaps in the existing literature. Although the term “rapid” is used in the original definition, scoping reviews can be arduous as identifying some types of literature e.g. grey literature, can be challenging.  Arksey & O’Malley’s initial outline is further elaborated by Levac. Quality appraisal is an optional element in scoping studies. A 6th step, the consultation exercise, can act as a knowledge translation component to the review, and although advocated it is not often reported.

Although scoping studies are not QES per se, given the diverse nature of data included, authors often use narrative methods to present findings.  As such, findings tend to be more aggregative than interpretative, but this can vary widely according to the research question.

Starting Out in QES: Some Reflections

As someone medically trained and familiar with traditional systematic review, engaging with QES methodologies can be intimidating. For example, the idea of refining the research question whilst iteratively searching the literature can feel disorderly.

Quality appraisal of studies is challenging, given the wide variety and heterogeneity of studies that may be included. How does one account for a study, which may be methodologically flawed but contests or reveals unique insights about the phenomena under scrutiny?

Working in a multidisciplinary team, with colleagues experienced in qualitative research is essential. The principles of qualitative research remain constant across the methodologies, the nuances are fun to learn.

Being reflexive can be uncomfortable.

Synthesis is messy, time consuming and fuzzy – from all of this somehow clarity will materialize (having faith in that is very uncomfortable!).

BUT the process is intellectually stimulating and the product, hopefully, has potential to make sense of ambiguous but central research topics in humanism.

Interested in learning more about QES?