|
Getting your Trinity Audio player ready...
|
Contents
- Glossary of Key Terms
- Key Takeaways
- What Is a Realist Review?
- The Philosophical Foundations of Realist Reviews
- The CMO Configuration: Context, Mechanism, and Outcome
- When Should You Use a Realist Review?
- The Steps of Conducting a Realist Review
- RAMESES Standards for Realist Reviews
- Strengths and Limitations of Realist Reviews
- How Does a Realist Review Handle Evidence Quality?
- Realist Reviews in Health and Social Care Research
- Realist Review Versus Realist Evaluation: What Is the Difference?
- Reporting and Publishing a Realist Review
- Practical Challenges and How to Address Them
- Team Composition and Skills for a Realist Review
- Frequently Asked Questions
Glossary of Key Terms
| Term | Definition |
| Causal Mechanism | The underlying process or reasoning that explains why an intervention produces an outcome under certain conditions. |
| CMO Configuration | Context plus Mechanism plus Outcome: the core analytical unit of a realist review. |
| Context | The social, institutional, cultural, or temporal setting in which an intervention operates. |
| Demi-regularities | Patterns in data that are not universal laws but recur with enough frequency to suggest an underlying mechanism. |
| Initial Program Theory | A preliminary explanatory model of how and why an intervention is expected to work, developed before data synthesis. |
| Mechanism | The process or resource that is triggered by an intervention and produces change in participants. |
| Middle-Range Theory | A theory operating between grand theory and raw empiricism, explaining patterns without claiming universal applicability. |
| Outcome | The measurable change or effect produced when a mechanism is triggered within a specific context. |
| Program Theory | An overarching explanatory account of how, why, for whom, and under what circumstances an intervention produces outcomes. |
| Purposive Sampling | A deliberate, theory-driven approach to selecting sources based on their relevance to explaining mechanisms. |
| Realist Evaluation | A primary research methodology aligned with realist philosophy, focused on understanding what works, for whom, and why. |
| Realist Synthesis | An alternative term for realist review, emphasizing the active process of integrating and building theory from evidence. |
| Retroduction | A reasoning process that works backward from observed outcomes to identify plausible underlying mechanisms. |
| Stakeholder | Anyone with a vested interest in an intervention, including policymakers, practitioners, beneficiaries, and researchers. |
| Transferability | The degree to which findings from a realist review can be applied to different contexts or settings. |
Key Takeaways
- A realist review is a theory-driven systematic review method that asks what works, for whom, in what circumstances, and why, rather than simply whether an intervention works.
- The CMO (Context, Mechanism, Outcome) configuration is the central analytical tool, explaining the conditions under which mechanisms are triggered to produce outcomes.
- Unlike traditional systematic reviews, realist reviews welcome heterogeneous evidence, including qualitative data, grey literature, and stakeholder insight, to build and test program theory.
- Realist reviews are especially valuable for evaluating complex social interventions where context is inseparable from effect.
What Is a Realist Review?
A realist review is a theory-driven, interpretive approach to synthesizing research evidence. It seeks to explain how interventions work by identifying the underlying mechanisms, the contexts in which those mechanisms are activated, and the outcomes they produce. Developed primarily by Ray Pawson and colleagues, it is grounded in realist philosophy of science and is designed to answer nuanced questions about complex social programs.
Traditional systematic reviews often ask: does this intervention work? A realist review asks a fundamentally different set of questions, captured in the phrase commonly attributed to Pawson: what works, for whom, in what circumstances, in what respects, and why?
How Does a Realist Review Differ from a Conventional Systematic Review?
A realist review differs from a conventional systematic review in philosophy, method, and purpose. Where conventional reviews aim to aggregate data to produce a pooled effect estimate, realist reviews aim to build and refine explanatory theory about how and why effects occur.
| Dimension | Conventional Systematic Review | Realist Review |
| Core question | Does it work? | Why does it work, for whom, and in what context? |
| Philosophical base | Positivism or empiricism | Critical realism |
| Evidence inclusion | Narrow, defined by study design | Broad, theory-driven purposive sampling |
| Primary output | Effect size or pooled estimate | Refined program theory |
| Heterogeneity | Treated as a problem | Treated as informative data |
| Qualitative evidence | Often excluded or peripheral | Central to mechanism identification |
| Role of context | Controlled or excluded | Integral to analysis |
The Philosophical Foundations of Realist Reviews
Realist reviews are grounded in critical realism, a philosophy of science associated with Roy Bhaskar. Critical realism holds that reality exists independently of our observations, that causal mechanisms exist at multiple levels, and that the relationship between cause and effect is not linear but contingent on context.
What Is Critical Realism and Why Does It Matter?
Critical realism matters because it provides the ontological and epistemological foundation for the realist review’s distinctive logic. It holds that the world is stratified into three domains: the real (mechanisms), the actual (events), and the empirical (observations). Mechanisms exist whether or not they are observed, and they produce outcomes only when activated by the right contextual conditions.
| Domain | Description | Example |
| Real | Mechanisms that exist and have causal power | A financial incentive creates motivation |
| Actual | Events that occur when mechanisms are triggered | Participants attend more sessions |
| Empirical | What is observed and measured | Outcome data collected in a study |
Middle-range theory is the explanatory level at which realist reviews operate. It is specific enough to be testable with evidence, yet general enough to be transferable across settings. Rather than producing grand theories about human behavior, realist reviews aim to produce explanations at this intermediate level that can guide policy and practice.
The CMO Configuration: Context, Mechanism, and Outcome
The CMO configuration is the fundamental unit of analysis in a realist review. It specifies that outcomes are produced when a mechanism is triggered within a particular context. No mechanism produces an outcome in isolation; context always shapes whether and how mechanisms fire.
What is Context in a Realist Review?
Context refers to any feature of the setting that influences whether a mechanism is activated. Context is not merely the backdrop to an intervention; it is an active ingredient in producing outcomes.
- Social context: cultural norms, relationships, trust levels, peer influence
- Institutional context: organizational culture, management support, resource availability
- Economic context: funding structures, socioeconomic conditions of participants
- Geographic context: urban versus rural settings, access to services
- Temporal context: historical moment, implementation phase, policy environment
What are Mechanisms in a Realist Review?
Mechanisms are the processes by which an intervention brings about change. They are often invisible in outcome data and must be inferred. A mechanism has two components: a resource introduced by the intervention and the reasoning or response that resource triggers in participants.
| Mechanism Component | Description | Example |
| Resource | What the intervention provides or introduces | Financial bonus for attendance |
| Reasoning | How participants respond to the resource | Participants feel valued and reciprocate with effort |
| Resulting behavior | The action or change that follows | Increased attendance rates |
What are Outcomes in a Realist Review?
Outcomes in a realist review are not simply end points. They include both intended and unintended consequences, short and long-term effects, and intermediate states. Realist reviews often distinguish between proximal outcomes (immediate effects) and distal outcomes (longer-term impacts).
When Should You Use a Realist Review?
A realist review is best suited to situations where a simple yes-or-no verdict on an intervention is insufficient. It is the preferred method when complexity, context-dependence, and theoretical understanding are priorities.
Indicators That a Realist Review Is Appropriate
- The intervention is complex and involves multiple interacting components
- Evidence on the intervention is heterogeneous and difficult to pool statistically
- Previous reviews have produced inconsistent findings across settings
- Policymakers need to understand why effects vary, not just whether they occur
- The intervention targets social, behavioral, or system-level change
- Qualitative and mixed-methods evidence is abundant and relevant
- The goal is to transfer lessons from one context to another
| Intervention Type | Suitable for Realist Review? | Reason |
| Complex public health programs | Yes | Multiple actors, contexts, and pathways |
| Drug efficacy in clinical trials | No | Mechanism is pharmacological, context less variable |
| School-based prevention programs | Yes | Highly context-dependent outcomes |
| Surgical technique comparison | No | Controlled conditions, limited social mechanism |
| Community mental health services | Yes | Relationship-dependent, variable settings |
| Vaccine immunogenicity | No | Biological mechanism, not social |
The Steps of Conducting a Realist Review
A realist review follows a structured but iterative process. The steps are not strictly linear; researchers cycle back to refine theory as evidence accumulates. The process was formalized in Pawson’s 2006 framework and later elaborated through RAMESES (Realist And Meta-narrative Evidence Syntheses: Evolving Standards) guidance.
Step 1: Clarify the Scope and Purpose
- Define the intervention or program to be reviewed
- Identify the policy or practice question driving the review
- Consult stakeholders to understand real-world relevance
- Determine the level of theory to be developed or tested: grand, middle-range, or substantive
Step 2: Develop the Initial Program Theory
Before searching for evidence, the review team develops a preliminary account of how the intervention is assumed to work. This is the initial program theory, often drawn from published theory, logic models, stakeholder interviews, or prior evaluations.
- Map assumed causal pathways from inputs to outcomes
- Identify candidate mechanisms that may explain effects
- Hypothesize which contexts might activate or suppress those mechanisms
- Document CMO configurations as propositions to be tested against evidence
Step 3: Search for Evidence
Searching in a realist review is purposive and theory-driven rather than exhaustive. The goal is to find evidence that can confirm, refute, or refine the program theory, not to retrieve every study on the topic.
| Source Type | Purpose in Realist Review |
| Randomized controlled trials | Provide outcome data; help test whether mechanisms produce expected effects |
| Qualitative studies | Reveal participant reasoning and context-specific experiences |
| Process evaluations | Illuminate implementation mechanisms and fidelity |
| Grey literature | Capture practitioner knowledge and policy documents |
| Stakeholder interviews | Elicit tacit knowledge about how programs work in practice |
| Theory papers | Provide conceptual scaffolding for mechanism identification |
Step 4: Assess Relevance and Rigor
Unlike conventional systematic reviews, realist reviews do not exclude studies solely based on study design. Quality assessment asks: is this source relevant to our program theory, and is it rigorous enough to trust the inference it supports?
- Relevance: does the study provide data about context, mechanism, or outcome in the CMO configuration?
- Rigor: is the study sufficiently trustworthy to base inferences upon, given its own methodology’s standards?
Step 5: Extract and Synthesize Data
Data extraction focuses on identifying evidence relevant to CMO configurations. Synthesis involves comparing, contrasting, and integrating findings to test and refine the program theory.
- Identify data segments that describe contexts, mechanisms, or outcomes
- Code segments according to CMO components
- Look for demi-regularities: patterns that recur across contexts
- Use retroduction to reason from observed patterns to underlying mechanisms
- Integrate evidence to produce refined CMO configurations
Step 6: Refine the Program Theory
The final stage involves consolidating the evidence into a revised, evidence-informed program theory. This theory makes explicit statements about which mechanisms are activated by what resources, in which contexts, to produce which outcomes.
RAMESES Standards for Realist Reviews
The RAMESES project developed reporting and quality standards for realist reviews to improve transparency and replicability. These standards cover both the conduct and the reporting of realist reviews.
What Are the RAMESES Reporting Standards?
RAMESES reporting standards are a set of quality criteria published to guide authors in documenting every stage of a realist review. They ensure that readers can understand the theoretical basis, search strategy, synthesis logic, and conclusions of the review.
| RAMESES Standard Area | Key Requirements |
| Background and rationale | Justify the realist approach; state the program theory to be tested |
| Objectives | Specify what the review aims to explain using CMO language |
| Search strategy | Describe purposive sampling logic; document sources and iteration |
| Quality appraisal | Explain criteria for relevance and rigor for each source type |
| Data extraction | Describe how CMO data were identified and coded |
| Synthesis | Explain how evidence was compared and integrated to refine theory |
| Findings | Present refined CMO configurations with supporting evidence |
| Discussion | Assess transferability; identify gaps and boundary conditions |
Strengths and Limitations of Realist Reviews
| Strengths | Limitations |
| Explains why interventions work, not just whether they do | More resource-intensive than narrative or meta-analytic reviews |
| Accommodates heterogeneous and mixed-method evidence | Requires substantial expertise in theory development |
| Produces actionable knowledge for context-specific implementation | Program theory is often implicit and difficult to articulate |
| Identifies conditions under which effects do or do not occur | Boundaries of evidence inclusion can be difficult to define and defend |
| Engages stakeholders, improving relevance and uptake | Replication and standardization are challenging |
| Transfers learning across settings through refined theory | Risk of confirmation bias in purposive searching if poorly managed |
How Does a Realist Review Handle Evidence Quality?
Evidence quality in a realist review is assessed differently from conventional reviews. Rather than ranking studies by design hierarchy, the realist approach evaluates each piece of evidence by whether it is fit for purpose: can it tell us something reliable about a context, mechanism, or outcome within the program theory?
Criteria for Judging Evidence in a Realist Review
| Criterion | Conventional Review | Realist Review |
| Relevance | Defined by PICO and study design | Defined by contribution to CMO understanding |
| Rigor | Assessed by universal quality scales | Assessed relative to the study’s own methodology |
| Inclusion logic | Hierarchical (RCT preferred) | Purposive and theory-driven |
| Exclusion criteria | Rigid and pre-specified | Iterative; may change as theory develops |
Realist Reviews in Health and Social Care Research
Realist reviews have been most widely applied in health and social care research, where interventions are complex, implementation is variable, and context profoundly shapes outcomes. They are increasingly used in public health, nursing, social work, and education research.
Common Application Areas
- Public health programs: smoking cessation, obesity prevention, alcohol interventions
- Mental health services: community care, peer support, crisis intervention
- Primary care: case management, patient self-management programs
- Social care: looked-after children, elder care, welfare programs
- Criminal justice: rehabilitation, restorative justice, recidivism reduction
- Education: mentoring, dropout prevention, school leadership programs
- International development: aid program evaluation, capacity building
What Makes Realist Reviews Valuable in Policy Contexts?
Realist reviews are valuable in policy contexts because they move beyond verdict-based evidence to produce transferable, context-sensitive guidance. Policymakers can use refined CMO configurations to understand not just whether to implement an intervention but how to adapt it for their own setting.
- They identify the conditions necessary for success rather than prescribing a single model
- They explain why the same program produces different results in different places
- They surface unintended consequences that aggregate data can obscure
- They build cumulative knowledge across a field rather than producing isolated verdicts
Realist Review Versus Realist Evaluation: What Is the Difference?
Realist evaluation and realist review share the same philosophical foundation but differ in their relationship to primary data. Realist evaluation is a primary research methodology in which investigators collect new data to test program theory. A realist review synthesizes existing evidence; it does not collect new data.
| Feature | Realist Evaluation | Realist Review |
| Data source | Newly collected primary data | Existing published and grey literature |
| Role of researcher | Investigator in the field | Synthesizer of existing knowledge |
| Scope | Single program or site | Multiple programs, sites, contexts |
| Typical output | Theory tested against one program | Refined theory tested across many programs |
| Resource requirements | High fieldwork costs | High literature search and synthesis costs |
Reporting and Publishing a Realist Review
A completed realist review is typically published with a protocol, a full review report, and often a summary for policymakers. Transparent reporting is essential for readers to evaluate the credibility of the program theory produced.
Key Elements of a Realist Review Report
| Report Section | Content |
| Abstract | Objectives, methods, key CMO findings, policy implications |
| Background | Problem framing, rationale for realist approach, initial program theory |
| Methods | Search strategy, evidence appraisal criteria, synthesis approach |
| Findings | Refined CMO configurations with supporting evidence from multiple sources |
| Discussion | Transferability, limitations, boundary conditions, theoretical contribution |
| Conclusions | Practice and policy implications; priority areas for further research |
| Supplementary materials | Search strategies, data extraction forms, stakeholder engagement log |
Practical Challenges and How to Address Them
| Challenge | Description | Suggested Response |
| Articulating the initial program theory | Theory may be implicit in policy documents or practitioner knowledge | Use stakeholder interviews and logic model exercises early in the process |
| Managing a large and heterogeneous evidence base | Purposive searching can still yield hundreds of sources | Apply iterative relevance screening tied explicitly to CMO propositions |
| Maintaining transparency in synthesis | Reasoning from evidence to mechanism is interpretive | Use member-checking, audit trails, and team consensus processes |
| Engaging stakeholders meaningfully | Stakeholders may not understand realist concepts | Use plain-language summaries of CMO configurations for consultation |
Team Composition and Skills for a Realist Review
A realist review requires a multidisciplinary team because the method combines systematic searching, theoretical reasoning, and interpretive synthesis. No single researcher typically holds all necessary skills.
- Information specialist: conducts systematic and purposive database searches
- Subject matter expert: provides substantive knowledge of the intervention domain
- Methodologist: guides the realist synthesis process and quality appraisal
- Stakeholder representative: contributes practitioner or service user perspectives
- Statistician: optional, assists with interpreting quantitative outcome data
Frequently Asked Questions
What is the difference between a realist review and a systematic review?
A systematic review aims to aggregate evidence to answer whether an intervention works, typically through statistical pooling. A realist review is theory-driven and asks why, for whom, and under what circumstances an intervention works. Systematic reviews usually restrict evidence to high-quality experimental designs; realist reviews purposively sample any evidence that illuminates context, mechanism, or outcome. The output of a systematic review is usually an effect estimate; the output of a realist review is a refined program theory in the form of CMO configurations.
How long does a realist review take to complete?
A full realist review typically takes 12 to 24 months from inception to publication. This includes protocol development and stakeholder consultation (1 to 3 months), initial program theory development (1 to 2 months), searching and screening (2 to 4 months), data extraction and quality appraisal (3 to 6 months), synthesis and theory refinement (2 to 4 months), and write-up and peer review (2 to 4 months). Rapid realist reviews can be completed in 3 to 6 months but involve trade-offs in scope and depth.
Can a realist review include quantitative evidence?
Yes. Realist reviews can and often do include quantitative evidence, including data from randomized controlled trials, cohort studies, and administrative datasets. Quantitative data are used to test whether expected outcomes occur and in which contexts; they are synthesized alongside qualitative data to provide a richer picture of the CMO configuration. The realist approach does not privilege any evidence type but selects sources based on their relevance to the program theory.
What is a rapid realist review?
A rapid realist review follows the same philosophical principles as a full realist review but is conducted within a compressed timeframe, usually 3 to 6 months, to meet urgent policy or practice needs. It typically involves a narrower scope, more targeted searching, and a more focused program theory. Rapid realist reviews are transparent about their limitations and the trade-offs made, and they are increasingly common in health system decision-making, particularly in response to emerging issues or policy deadlines.
How do you develop a program theory for a realist review?
Program theory development begins before the formal search and involves three main activities:
- reviewing existing theory and conceptual literature related to the intervention;
- examining published logic models, policy documents, and prior evaluations; and
- consulting stakeholders including practitioners, policymakers, and service users.
The initial theory is expressed as a set of CMO propositions stating which mechanisms are expected to be activated by which contextual features to produce which outcomes. The theory is then iteratively tested and refined as evidence is gathered.
What databases and sources are searched in a realist review?
Searching in a realist review is purposive and iterative rather than exhaustive. Common databases searched include MEDLINE, Embase, PsycINFO, CINAHL, Cochrane, ERIC, and Web of Science depending on the topic. Grey literature sources such as government websites, policy repositories, organizational reports, and practitioner publications are also searched. Citation chaining, reference list scanning, and expert consultation are frequently used to identify theory papers and evaluations not indexed in standard databases.
Is a realist review the same as a meta-ethnography?
No. A realist review and a meta-ethnography are both qualitative synthesis methods but differ in purpose and process. Meta-ethnography, developed by Noblit and Hare, synthesizes qualitative studies through conceptual translation to produce new interpretations; it does not require a program theory or CMO framework. A realist review is explicitly theory-driven and integrates both qualitative and quantitative evidence around CMO configurations. Both are appropriate for complex interventions but answer different types of questions.
How do realist reviews contribute to evidence-based practice?
Realist reviews contribute to evidence-based practice by producing actionable, context-sensitive guidance that goes beyond simple effect estimates. Rather than recommending that practitioners adopt or abandon an intervention outright, a realist review specifies the conditions under which an intervention is likely to succeed. This allows practitioners to assess whether their own context matches the conditions identified in the review, adapt implementation accordingly, and make informed decisions about which aspects of an intervention are essential to preserve and which can be modified.
Comment