Article type
Abstract
Background:
Randomized controlled trials (RCTs) may not address all key questions regarding the balance of benefits and harms of therapeutic interventions, while nonrandomized studies of intervention (NRSIs) can provide complementary, sequential, or replacement evidence for RCTs in evidence syntheses. Interest in including RCTs and NRSIs for decision-making is growing.
Objectives:
To summarize when NRSIs are needed in evidence synthesis of RCTs, to explore how NRSIs can be integrated into a meta-analysis of RCTs, and to investigate the impact on estimates of pooled bodies of evidence when NRSIs are included.
Methods:
PubMed was searched for eligible systematic reviews that included RCTs and NRSIs under the same outcome. The clinical scenarios for considering NRSIs in evidence synthesis were grouped into 4 categories, and the meta-analytic methods were examined to clarify how RCTs and NRSIs were combined. The impact of adding data from NRSIs to the meta-analysis of RCTs was further explored using both the traditional random-effects model and advanced statistical methods (eg, bias-corrected meta-analysis model).
Results:
In total, 220 systematic reviews were included in the analysis. The main justification for including NRSIs was safety/rare outcomes (n = 140, 63.6%), long-term outcomes (n = 36, 16.4%), generalization (n = 11, 5.0%), and others (n = 33, 15.0%). Specifically, 203 (92.3%) pooled RCTs and NRSIs in the same meta-analysis, 167 (75.9%) combined the crude estimates of NRSIs, and 25 (11.9%) directly combined the results without distinguishing different NRSI types. Compared with conclusions drawn from RCTs, the inclusion of NRSIs led to 35.8% of meta-analyses changing the significance. After employing the bias-corrected model, 30.3% of studies showed opposite results than those using the traditional meta-analysis model. When the RCTs and NRSIs were analyzed separately with inconsistent results, over half (58.8%) of the reviews reported positive conclusions only.
Conclusions:
Systematic reviews usually considered NRSIs in the context of adverse/rare outcomes, long-term outcomes, and generalizations. The conventional practice of directly integrating data from NRSIs into RCTs may produce misleading conclusions, as almost a third of meta-analyses change their conclusions after using advanced meta-analytic approaches. Suggestions proposed here will help provide comprehensive guidelines for integrating evidence from RCTs and NRSIs.
Randomized controlled trials (RCTs) may not address all key questions regarding the balance of benefits and harms of therapeutic interventions, while nonrandomized studies of intervention (NRSIs) can provide complementary, sequential, or replacement evidence for RCTs in evidence syntheses. Interest in including RCTs and NRSIs for decision-making is growing.
Objectives:
To summarize when NRSIs are needed in evidence synthesis of RCTs, to explore how NRSIs can be integrated into a meta-analysis of RCTs, and to investigate the impact on estimates of pooled bodies of evidence when NRSIs are included.
Methods:
PubMed was searched for eligible systematic reviews that included RCTs and NRSIs under the same outcome. The clinical scenarios for considering NRSIs in evidence synthesis were grouped into 4 categories, and the meta-analytic methods were examined to clarify how RCTs and NRSIs were combined. The impact of adding data from NRSIs to the meta-analysis of RCTs was further explored using both the traditional random-effects model and advanced statistical methods (eg, bias-corrected meta-analysis model).
Results:
In total, 220 systematic reviews were included in the analysis. The main justification for including NRSIs was safety/rare outcomes (n = 140, 63.6%), long-term outcomes (n = 36, 16.4%), generalization (n = 11, 5.0%), and others (n = 33, 15.0%). Specifically, 203 (92.3%) pooled RCTs and NRSIs in the same meta-analysis, 167 (75.9%) combined the crude estimates of NRSIs, and 25 (11.9%) directly combined the results without distinguishing different NRSI types. Compared with conclusions drawn from RCTs, the inclusion of NRSIs led to 35.8% of meta-analyses changing the significance. After employing the bias-corrected model, 30.3% of studies showed opposite results than those using the traditional meta-analysis model. When the RCTs and NRSIs were analyzed separately with inconsistent results, over half (58.8%) of the reviews reported positive conclusions only.
Conclusions:
Systematic reviews usually considered NRSIs in the context of adverse/rare outcomes, long-term outcomes, and generalizations. The conventional practice of directly integrating data from NRSIs into RCTs may produce misleading conclusions, as almost a third of meta-analyses change their conclusions after using advanced meta-analytic approaches. Suggestions proposed here will help provide comprehensive guidelines for integrating evidence from RCTs and NRSIs.