To what extent do trials stopped early for benefit overesti­mate treatment effects?

Article type
Authors
Guyatt G, Bassler D, Montori V, Briel M, Lane M, Zhou Q, Glasziou P
Abstract
Background: Modeling studies suggest that trials stopped early for benefit systematically overestimate treatment effects, and this overestimate may be very large. A systematic review of such trials provided confirmatory evidence of large overestimates of effect, and suggested that trials stopped early for benefit are becoming more common. Objectives: To quantify the degree of any overestimation of effects seen in trials stopped early for benefit and to identify determinants of the magnitude of overestimation. Methods: We systematically searched for all trials stopped early for benefit up to September 2007. For each stopped early trial we searched for systematic reviews of trials addressing the same clinical question, and updated these where necessary. We compared the magnitude of effect in the stopped early trials to the magnitude of effect in the other trials included in the systematic reviews addressing the same question. Each study yielded a summary statistic (logRR) and an associated variance that provided weights for a meta-regression to evaluate the determinants of the estimated treatment effect. We explored determinants of the magnitude of the difference between the stopped early trial and the combined other trial estimates, including: the quality of the stopping rule; the trial methodological quality; and the total number of events. Results: We identified 195 stopped early trials which led to the identification of 238 potentially corresponding systematic reviews. Of these, 71 reviews associated with 64 separate questions ultimately proved eligible. For the 71 stopped early trials included in the analysis, we used 476 corresponding other trials; a range of 1 to 3 stopped trials, and 1 to 44 other trials per question. The results of the analysis will be available for presentation at the Singapore Colloquium.