Article type
Year
Abstract
Background: Little guidance for addressing missing participant outcome data in meta-analyses and practice guidelines is available.
Objectives: To explore the use of decision thresholds to address risk of bias associated with missing binary outcome data.
Methods: We applied the GRADE approach to missing data. We initially conducted a complete case analysis, and then conducted progressively more stringent sensitivity analyses imputing outcomes for those with missing outcomes in each study.
Results: Rather than rating down using a threshold of no effect, one may choose a decision threshold representing the smallest difference patients would consider important. Consider, for instance, probiotics for the prevention of Clostridium difficile infection (CDI) (Johnston 2012). In 13 of 20 included randomized trials, data on CDI were missing for 5% to 45% of participants across studies. For the control group, we assumed that the event rate in participants with missing data was the same as the event rate in participants who were successfully followed. For the probiotic group, we recalculated pooled treatment effects by assuming the following risk incidence (RI) in participants with missing data compared with those who were successfully followed: RILTFU/FU 1.5, 2.0, 3.0 and 5.0. Using a threshold of relative risk of 1.0, our results proved robust to even the most extreme assumption. However, patients are likely to decline treatment if the benefit of probiotics is sufficiently small (say 2%). Given a risk of CDI of 5.1% without probiotics, the absolute risk reduction of 3.6% (95% confidence interval (CI) 2.4% to 4.7%) in the complete case analysis, decreases to 2.8% (95% CI 1.6% to 4.1%) with a RILTFU/FU of 5.0. Given that the lower boundary of the 95% CI now crosses our threshold of 2%, one would rate down for risk of bias.
Conclusions: Since choosing a decision threshold other than no effect involves a value judgment, this approach may be best applied in the context of practice guidelines.
Johnston BC, et al. Probiotics for the prevention of Clostridium difficile-associated diarrhea: a systematic review and meta-analysis. Ann Intern Med. 2012;157(12):878-88.
Objectives: To explore the use of decision thresholds to address risk of bias associated with missing binary outcome data.
Methods: We applied the GRADE approach to missing data. We initially conducted a complete case analysis, and then conducted progressively more stringent sensitivity analyses imputing outcomes for those with missing outcomes in each study.
Results: Rather than rating down using a threshold of no effect, one may choose a decision threshold representing the smallest difference patients would consider important. Consider, for instance, probiotics for the prevention of Clostridium difficile infection (CDI) (Johnston 2012). In 13 of 20 included randomized trials, data on CDI were missing for 5% to 45% of participants across studies. For the control group, we assumed that the event rate in participants with missing data was the same as the event rate in participants who were successfully followed. For the probiotic group, we recalculated pooled treatment effects by assuming the following risk incidence (RI) in participants with missing data compared with those who were successfully followed: RILTFU/FU 1.5, 2.0, 3.0 and 5.0. Using a threshold of relative risk of 1.0, our results proved robust to even the most extreme assumption. However, patients are likely to decline treatment if the benefit of probiotics is sufficiently small (say 2%). Given a risk of CDI of 5.1% without probiotics, the absolute risk reduction of 3.6% (95% confidence interval (CI) 2.4% to 4.7%) in the complete case analysis, decreases to 2.8% (95% CI 1.6% to 4.1%) with a RILTFU/FU of 5.0. Given that the lower boundary of the 95% CI now crosses our threshold of 2%, one would rate down for risk of bias.
Conclusions: Since choosing a decision threshold other than no effect involves a value judgment, this approach may be best applied in the context of practice guidelines.
Johnston BC, et al. Probiotics for the prevention of Clostridium difficile-associated diarrhea: a systematic review and meta-analysis. Ann Intern Med. 2012;157(12):878-88.