Article type
Year
Abstract
Background: The potential impact of stopping studies early on estimates of effect in meta-analysis has been an area of controversy. We encountered this issue in a systematic review of supplemental oxygen in the prevention of surgical site infections (SSIs) site infections which are a common, potentially serious, and costly complication of surgery.
Methods: We conducted a comprehensive search to identify all published and unpublished randomized controlled trials (RCTs) evaluating the effect of supplemental perioperative oxygen on SSIs. Two reviewers independently determined study eligibility and extracted data.
Results: Seven RCTs that included a total of 4,544 patients met eligibility criteria. Studies suffered from moderate risk of bias. Three out of seven (43%) trials were stopped early: one for benefit (apparent 54% reduction in relative risk), one for harm (apparent doubling of risk), and one for futility. The mean number of SSIs was 129 (range5-272) in the non-stopped and 32 (range 27-41) in the stopped trials. Confidence intervals around the pooled estimate from all trials included a 35% reduction, and a 23% increase, in relative risk (614events; RR 0.89 95% CI 0.65 to 1.23), with substantial heterogeneity (I2 = 68%) (Figure1). Differences in patients, interventions, outcome measurement, or risk of bias could not explain differences in study results.
Conclusions: Our systematic review demonstrates how trials stopped early for benefit or harm tend to provide misleading results. Stopping early for any reason reduces the likelihood of accumulating sufficient evidence to provide definitive results.
Methods: We conducted a comprehensive search to identify all published and unpublished randomized controlled trials (RCTs) evaluating the effect of supplemental perioperative oxygen on SSIs. Two reviewers independently determined study eligibility and extracted data.
Results: Seven RCTs that included a total of 4,544 patients met eligibility criteria. Studies suffered from moderate risk of bias. Three out of seven (43%) trials were stopped early: one for benefit (apparent 54% reduction in relative risk), one for harm (apparent doubling of risk), and one for futility. The mean number of SSIs was 129 (range5-272) in the non-stopped and 32 (range 27-41) in the stopped trials. Confidence intervals around the pooled estimate from all trials included a 35% reduction, and a 23% increase, in relative risk (614events; RR 0.89 95% CI 0.65 to 1.23), with substantial heterogeneity (I2 = 68%) (Figure1). Differences in patients, interventions, outcome measurement, or risk of bias could not explain differences in study results.
Conclusions: Our systematic review demonstrates how trials stopped early for benefit or harm tend to provide misleading results. Stopping early for any reason reduces the likelihood of accumulating sufficient evidence to provide definitive results.
Images
