Article type
Year
Abstract
Background: Truncated randomized controlled trials (tRCTs) stopped early for benefit often show implausibly large treatment effects1. Systematic reviews and meta-analyses including such trials may therefore yield biased results.
Objectives: To examine if and how authors of systematic reviews and meta-analyses that include tRCTs address the potential bias introduced by tRCTs and to determine the weight of tRCTs on pooled results for the outcomes that led to the early termination.
Methods: We searched The Cochrane Library and MEDLINE for systematic reviews that included at least one of 160 previously identified tRCTs and extracted data on the approaches the authors used to handle the potential bias and on the impact of tRCTs in systematic reviews and meta-analyses.
Results: We identified 97 systematic reviews that included at least one tRCT of which 43 (44%) included more than one tRCT. Of the 97 systematic reviews, 68 (70%) did not mention truncation of the included tRCTs at all. Methodological quality was assessed through use of individual criteria in 57/97(59%) systematic reviews and only two of these 57 reported early stopping for benefit as one criterion. Seventy of the 97 systematic reviews (72%) included the tRCT in at least one meta-analysis for the outcome that led to the early termination and only four of these 70 explored truncation as a potential explanation for heterogeneity by performing subgroup analysis. The reported weight attributed to the tRCT(s) in 165 meta-analyses for the outcome that led to the early termination of the tRCTs included in the 70 systematic reviews was > 40% in 76/165 (46%) of the meta-analyses.
Conclusions: Most systematic reviews and meta-analyses including tRCTs fail to consider, in any way, the possible overestimates of effect that may result from trials stopped early for benefit. This bias may, in some cases, be substantial.
References
1. Montori VM, Devereaux PJ, Adhikari NK, Burns KE, Eggert CH, Briel M, et al. Randomized trials stopped early for benefit: a systematic
review. JAMA 2005; 294:2203-9.
Objectives: To examine if and how authors of systematic reviews and meta-analyses that include tRCTs address the potential bias introduced by tRCTs and to determine the weight of tRCTs on pooled results for the outcomes that led to the early termination.
Methods: We searched The Cochrane Library and MEDLINE for systematic reviews that included at least one of 160 previously identified tRCTs and extracted data on the approaches the authors used to handle the potential bias and on the impact of tRCTs in systematic reviews and meta-analyses.
Results: We identified 97 systematic reviews that included at least one tRCT of which 43 (44%) included more than one tRCT. Of the 97 systematic reviews, 68 (70%) did not mention truncation of the included tRCTs at all. Methodological quality was assessed through use of individual criteria in 57/97(59%) systematic reviews and only two of these 57 reported early stopping for benefit as one criterion. Seventy of the 97 systematic reviews (72%) included the tRCT in at least one meta-analysis for the outcome that led to the early termination and only four of these 70 explored truncation as a potential explanation for heterogeneity by performing subgroup analysis. The reported weight attributed to the tRCT(s) in 165 meta-analyses for the outcome that led to the early termination of the tRCTs included in the 70 systematic reviews was > 40% in 76/165 (46%) of the meta-analyses.
Conclusions: Most systematic reviews and meta-analyses including tRCTs fail to consider, in any way, the possible overestimates of effect that may result from trials stopped early for benefit. This bias may, in some cases, be substantial.
References
1. Montori VM, Devereaux PJ, Adhikari NK, Burns KE, Eggert CH, Briel M, et al. Randomized trials stopped early for benefit: a systematic
review. JAMA 2005; 294:2203-9.