Understanding the process and impact of within-study selective reporting bias for harm outcomes (ORBIT II—outcome reporting bias in trials)

Saini P1, Gamble C1, Loke Y2, Altman DG3, Williamson P1, Kirkham JJ1
1University of Liverpool, 2University of East Anglia, 3University of Oxford

Background: The prevalence and impact of outcome reporting bias (ORB), whereby outcomes are selected for publication on the basis of the result, has previously been quantified for benefit outcomes in randomised controlled trials (RCTs) on a cohort of Cochrane systematic reviews. Important harm outcomes may also be subject to ORB where trialists prefer to focus on the positive benefits of an intervention. Empirical evidence suggests that the reporting of harms data is likely to be less complete than that of efficacy measures.

Objectives: • To estimate the prevalence of selective outcome reporting of harm outcomes in a cohort of both Cochrane and non-Cochrane Reviews • To consider the assessment of selective reporting of harm outcomes in both RCTs and non-randomised studies (NRSs) • To investigate the impact of ORB on the benefit-harm ratio • To understand the mechanisms that may lead to incomplete reporting of harms data.

Methods: A classification system for detecting ORB for harm outcomes in RCTs and NRSs was developed and applied to both a cohort of Cochrane systematic reviews and reviews identified via the Cochrane Database of Systematic Reviews (CDSR) and Database of Abstracts of Reviews (DARE). In a subset of reviews where ORB is identified, benefit-harm ratios will be calculated based on (i) the original analyses reported in the review (ii) the adjusted estimates once ORB has been accounted for using a suitable sensitivity adjustment method.

Results: From 2007 onwards a total of 234 reviews were identified from the CDSR and DARE databases in which approximately three quarters showed evidence of ORB. Full study findings from the study will be presented at the conference.

Conclusions: Making informed decisions that consider both benefits and harms of an intervention in an unbiased way is essential in order to make reliable benefit-harm predictions.