Article type
Year
Abstract
Background: Systematic reviews utilize randomized controlled trials (RCTs) to evaluate competing interventions. It has been suggested that RCTs are limited by low generalizability due to strict inclusion criteria. While non-RCTs have broader generalizability, they also have limitations. The added value of including them is unclear.
Objectives: To assess the added value of non-RCTs in a review of atypical antipsychotics (AAPs).
Methods: Review One consists only of RCTs to evaluate the comparative effectiveness of AAPs for schizophrenia while Review Two adds non-RCTs. The consistency, strength and direction of findings were compared.
Results: Review One found 52 head-to-head RCTs. This body of evidence has moderate internal validity and low external validity, with only three RCTs qualifying as effectiveness trials. Review Two added 101 non-RCTs. The bulk of the effectiveness evidence from non-RCTs was for the comparison of olanzapine and risperidone (24 studies). One-third was poor quality for various reasons, including lack of: inception cohorts, comparison of groups at baseline in cohorts or establishment of a stable baseline in before-after studies, and control for confounders. Seventy-two per cent were inadequate for making comparisons among the AAPs due to lack of direct comparisons; differences in definition and/or ascertainment method of outcomes prevented indirect analysis. This body of evidence has higher external validity, and lower internal validity. The findings on efficacy were similar to the RCT findings. For some outcomes non-RCT studies conflicted with the findings of RCTs, for example RCTs found a difference in cholesterol and glucose levels by AAP while observational studies did not. Non-RCTs reported good comparative evidence on serious harms, filling gaps left by RCTs, and confirming findings of RCTs on relative weight gain.
Conclusions: The marginal value of non-RCT studies when added to a large pool of RCTs assessing the comparative effectiveness of AAPs was limited mainly to adverse events.
Objectives: To assess the added value of non-RCTs in a review of atypical antipsychotics (AAPs).
Methods: Review One consists only of RCTs to evaluate the comparative effectiveness of AAPs for schizophrenia while Review Two adds non-RCTs. The consistency, strength and direction of findings were compared.
Results: Review One found 52 head-to-head RCTs. This body of evidence has moderate internal validity and low external validity, with only three RCTs qualifying as effectiveness trials. Review Two added 101 non-RCTs. The bulk of the effectiveness evidence from non-RCTs was for the comparison of olanzapine and risperidone (24 studies). One-third was poor quality for various reasons, including lack of: inception cohorts, comparison of groups at baseline in cohorts or establishment of a stable baseline in before-after studies, and control for confounders. Seventy-two per cent were inadequate for making comparisons among the AAPs due to lack of direct comparisons; differences in definition and/or ascertainment method of outcomes prevented indirect analysis. This body of evidence has higher external validity, and lower internal validity. The findings on efficacy were similar to the RCT findings. For some outcomes non-RCT studies conflicted with the findings of RCTs, for example RCTs found a difference in cholesterol and glucose levels by AAP while observational studies did not. Non-RCTs reported good comparative evidence on serious harms, filling gaps left by RCTs, and confirming findings of RCTs on relative weight gain.
Conclusions: The marginal value of non-RCT studies when added to a large pool of RCTs assessing the comparative effectiveness of AAPs was limited mainly to adverse events.