Investigating outcome reporting bias in Cochrane Cystic Fibrosis and Genetic Disorder (CFGD) reviews

Article type
Authors
Dwan K1, Wiliamson P1, Gamble C1, Remmington T2, Jahnke N2, Kirkham J1
1Centre for Medical Statistics and Health Evaluation, The University of Liverpool, Liverpool, UK
2Reproductive and Developmental Medicine, The University of Liverpool, Liverpool, UK
Abstract
Background: Outcome reporting bias (ORB) has been identified as a threat to the validity of evidence-based-medicine. Trial outcomes with statistically significant results are more likely to be published than non-significant outcomes. The ORBIT study investigated ORB in Cochrane reviews (Kirkham et al, 2010a) but only considered primary outcomes and included only two CFGD reviews. The prevalence and impact of ORB in CFGD reviews is unknown. Objectives: To assess ORB in primary and secondary outcomes in CFGD reviews and included RCTs; To assess the impact of ORB in a review using a sensitivity analysis; To compare review outcomes between protocol and review as in a recent paper by Kirkham et al, 2010b; To investigate the heterogeneity in the reporting of outcomes. Methods: Reviews published by the CFGD Group that identified at least one trial were included. This project followed the methodology developed during the ORBIT study where outcomes not reported or partially reported were classified based on the suspicion or ORB. Results: 82 reviews published prior to 2010 were included; 21 identified no RCTs. Sixty-one reviews were considered further containing 405 included trials and 21 trials excluded due to ‘no relevant outcome data’. Between protocol and review, there were three reviews that upgraded secondary outcomes to primary, 15 downgraded primary outcomes to secondary, two reviews included outcomes that were not in the protocol and two reviews excluded outcomes that were originally in the protocol. Nine reviews removed secondary outcomes between protocol and review and six reviews added secondary outcomes. Sixteen review protocols did not distinguish between primary and secondary outcomes. The assessment of ORB is ongoing at present. Conclusions: ORB is a problem in all areas of research. A core set of outcomes for genetic disorders will be an important step in reducing ORB and standardising outcome measures in these clinical areas.