Article type
Year
Abstract
Background: the Cochrane Handbook advises to judge the risk of bias (RoB) potentially introduced by selective reporting of outcome data (SORB) on the within-trial level, while judging if the outcomes expected to be reported in a given field are addressed.
Objectives: to summarize and discuss applied methods to judge and handle RoB due to selective reporting in Cochrane Reviews.
Methods: we obtained unique reviews from the Cochrane Musculoskeletal Group (CMSG) and the Cochrane Peripheral Vascular Diseases Group (PVD) published in 2011 to 2015, including at least 1 RCT. Extraction items included definitions used to judge the RoB by selective reporting, the frequency of scoring low, high or unclear RoB due to selective reporting, and the handling of this item in the 'Summary of findings' tables (SoF).
Results: Complete results will be presented at the Colloquium. Up to 2013, 162 reviews were identified, of which 42 PVD and 24 CMSG reviews were considered. The Figure shows that 83% of the reviews considered SORB. Of these, the majority of reviews referred to the Cochrane Handbook for the operationalization of how to assess SORB (Table). Most reviews assess SORB at the within-trial level, matching outcomes reported in protocols, trial registrations and methods sections to outcomes addressed in the results section. Only a minority (9%) followed the Handbook guidance to assess SORB at the within-trial and topic level; 17% focused on completeness of reporting of review-defined outcomes (review outcome level). SoF tables were provided in 44% of the included reviews (Figure). SORB was infrequently (15%) mentioned as a reason to downgrade the confidence in the estimate of the effect.
Conclusions: Cochrane guidance on SORB is insufficiently followed by CMSG and PVD reviewers. SORB is typically assessed on the within-trial level, without considering whether at least the main outcomes relevant to the topic or review question are addressed across trials. SORB seems to be given insufficient attention while GRADE-ing our confidence in the estimates of the effects. We suggest a stakeholder meeting to evaluate and discuss hurdles to assess SORB and to improve methods.
Objectives: to summarize and discuss applied methods to judge and handle RoB due to selective reporting in Cochrane Reviews.
Methods: we obtained unique reviews from the Cochrane Musculoskeletal Group (CMSG) and the Cochrane Peripheral Vascular Diseases Group (PVD) published in 2011 to 2015, including at least 1 RCT. Extraction items included definitions used to judge the RoB by selective reporting, the frequency of scoring low, high or unclear RoB due to selective reporting, and the handling of this item in the 'Summary of findings' tables (SoF).
Results: Complete results will be presented at the Colloquium. Up to 2013, 162 reviews were identified, of which 42 PVD and 24 CMSG reviews were considered. The Figure shows that 83% of the reviews considered SORB. Of these, the majority of reviews referred to the Cochrane Handbook for the operationalization of how to assess SORB (Table). Most reviews assess SORB at the within-trial level, matching outcomes reported in protocols, trial registrations and methods sections to outcomes addressed in the results section. Only a minority (9%) followed the Handbook guidance to assess SORB at the within-trial and topic level; 17% focused on completeness of reporting of review-defined outcomes (review outcome level). SoF tables were provided in 44% of the included reviews (Figure). SORB was infrequently (15%) mentioned as a reason to downgrade the confidence in the estimate of the effect.
Conclusions: Cochrane guidance on SORB is insufficiently followed by CMSG and PVD reviewers. SORB is typically assessed on the within-trial level, without considering whether at least the main outcomes relevant to the topic or review question are addressed across trials. SORB seems to be given insufficient attention while GRADE-ing our confidence in the estimates of the effects. We suggest a stakeholder meeting to evaluate and discuss hurdles to assess SORB and to improve methods.