Article type
Year
Abstract
Background: A previous study demonstrated that within-study selective non-reporting of outcomes can have a substantial effect on meta-analysis when the amount of missing data is large, however, in four of the five meta-analyses examined, the impact on conclusions was minimal due to the small amount of missing data1. The Cochrane Collaboration Steering Group has recently approved a larger MRC-funded project in this area.
Objectives: To estimate the prevalence and impact of within-study selective reporting in an unselected cohort of 300 reviews.
Methods: For each review, we will identify whether all eligible trials have been included in the meta-analysis of the primary efficacy outcome and produce a matrix showing the reporting of all outcomes in each trial, distinguishing full/partial/no information. The primary reviewer will be contacted, given information about our study, an outcome matrix for their review, and asked to participate by providing content expertise. For each trial not reporting the outcome of interest at all, the reviewer and research assistant will independently scrutinise all publications relating to that trial, classifying, according to an established protocol, how likely it was that the outcome had been measured. For each trial published within the last five years partially or not reporting the outcome of interest, the trialists will then be contacted to obtain the missing outcome data, if measured, and the reason for not reporting, or to confirm that the outcome was not measured.
Results: A pilot study of a random sample of 20 Cochrane reviews first published 2002-2004 suggests that 180 of the 300 reviews will include 860 trials not reporting any data for the primary outcome of the review and 540 trials partially reporting data.
Conclusions: This presentation will provide a forum for discussion of the proposed methodology with reviewers, to minimize potential problems during the study.
References
1. Williamson PR and Gamble C. Identification and impact of outcome selection bias in meta-analysis. Statistics in Medicine 2005; 24:1547-61.
Objectives: To estimate the prevalence and impact of within-study selective reporting in an unselected cohort of 300 reviews.
Methods: For each review, we will identify whether all eligible trials have been included in the meta-analysis of the primary efficacy outcome and produce a matrix showing the reporting of all outcomes in each trial, distinguishing full/partial/no information. The primary reviewer will be contacted, given information about our study, an outcome matrix for their review, and asked to participate by providing content expertise. For each trial not reporting the outcome of interest at all, the reviewer and research assistant will independently scrutinise all publications relating to that trial, classifying, according to an established protocol, how likely it was that the outcome had been measured. For each trial published within the last five years partially or not reporting the outcome of interest, the trialists will then be contacted to obtain the missing outcome data, if measured, and the reason for not reporting, or to confirm that the outcome was not measured.
Results: A pilot study of a random sample of 20 Cochrane reviews first published 2002-2004 suggests that 180 of the 300 reviews will include 860 trials not reporting any data for the primary outcome of the review and 540 trials partially reporting data.
Conclusions: This presentation will provide a forum for discussion of the proposed methodology with reviewers, to minimize potential problems during the study.
References
1. Williamson PR and Gamble C. Identification and impact of outcome selection bias in meta-analysis. Statistics in Medicine 2005; 24:1547-61.