The data was unobtainable. A survey of the reporting of missing data in a sample of Cochrane reviews

Article type
Authors
Struthers C1, Noel-Storr A1
1Cochrane Dementia and Cognitive Improvement Group, UK
Abstract
Background: It is widely acknowledged that publication bias is one of the most difficult to avoid in the preparation of systematic reviews. There are many factors that lead to data from clinical trials being unavailable for a meta-analysis, and this single source of bias significantly undermines the power of a Cochrane review to reflect the true picture of an intervention’s effectiveness.

Objectives: We decided to investigate the occurrence and extent of unobtainable results data in studies included in Cochrane reviews. We hope that our findings will lead to a more robust, proactive and standardized reporting of unavailable data that will reduce its occurrence, and thereby improve the quality of Cochrane reviews.

Methods: We developed a search strategy using terms designed to find all Cochrane reviews where data from trials eligible for inclusion were incomplete or unobtainable. We selected a random sample of 100 full reviews, and updated reviews. We also randomly selected Cochrane reviews with studies awaiting classification, or 'ongoing’ to see if that masked underlying publication bias.

Results: We analysed the reviews in both sample sets using a number of measures, including the proportion of eligible trials where the data were missing for the reasons outlined above. We also looked at the efforts made to obtain data, and how their absence was reported in the reviews.

Discussion: As part of the development of a Cochrane-wide citizen science initiative, we are considering how to include the tracking of results data in a portfolio of volunteer tasks. We are looking forward to a future where public involvement can bring pressure to bear on research authorities to make sure that all clinical trials are registered, protocols adhered to, and results data are made freely available for analysis by review authors and others in a timely manner.