Reporting, dealing with, and judging risk of bias associated with missing participant data in systematic reviews: a methodological survey

Tags: Oral
Akl EA1, Johnston B2, Carrasco A3, Brignardello Petersen R3, Neumann I3, Sun X3, Briel M4, Busse J3, Granados C3, Iorio A3, Irfan A5, Martinez L6, Mustafa RA3, Ramirez A7, Solá I6, Martinez Garcia L6, Selva A6, Sanabria AJ6, Zazueta OE6, Vernooij R6, Tikkinen KA8, Ebrahim S9, Vandvik PO10, Guyatt GH3, Alonso Coello P6
1American University of Beirut, Lebanon, 2SickKids Research Institute, Canada, 3McMaster University, Canada, 4University Hospital Basel, Switzerland, 5University of Illinois at Urbana-Champaign, USA, 6Iberoamerican Cochrane Centre, Spain, 7Central America Cochrane, Costa Rica, 8Tampere University Hospital, Finland, 9shanil.ebrahim@utoronto.ca, 10Norwegian Knowledge Centre for the Health Services, Norway

Background: When conducting a meta-analysis, systematic review authors need to deal with trials reporting missing participant data including an assessment of the risk of bias associated with missing participant data in the body of evidence.

Objectives: To describe how authors of systematic reviews are reporting, dealing with, and judging the risk of bias associated with missing data for dichotomous outcome.

Methods: We searched Medline and the Cochrane Database of Systematic Reviews for systematic reviews of randomized controlled trials published in 2010 and reporting a meta-analysis of dichotomous outcome. We randomly selected 101 Cochrane systematic reviews and 101 non-Cochrane systematic reviews. Teams of two reviewers selected eligible studies studies and abstracted data independently and in duplicate using standardized, piloted forms with accompanying written instructions. The Cochrane Collaboration Methods Innovation Fund funded this study.

Results: Of the 202 systematic reviews, 187 (93%) reported a standard meta-analysis, 166 (82%) addressed a medical topic, 130 (64%) assessed pharmacological interventions; they included a median of 5 trials (IQR 2–8). The table shows the percentage of systematic reviews that reported, dealt with, and judged the risk of bias associated with participant with missing data for dichotomous outcome. In general, Cochrane Reviews performed poorly (approximately 50% or less for all key criteria other than judging the risk of bias associated with missing participant data) and non-Cochrane Reviews very poorly (20% or less for all key criteria with the exception of 52% for judging risk of bias associated with missing participant data).

Conclusions: Both Cochrane and—particularly—non-Cochrane Reviews need to do far better in attending to issues of reporting and handling missing participant data.