Assessing publication bias in systematic reviews andmeta-analyses in the context of small n trials: application of a capture-recapture method

Article type
Authors
Coronado-Montoya S1, Kwakkenbos L2, Thombs B2
1Lady Davis Institute for Medical Research, Jewish General Hospital, Canada
2McGill University, Canada
Abstract
Background: Systematic reviews and meta-analyses should include published and non-published evidence. Evidence used to support recommendations for relatively new, easily-delivered, psychosocial interventions, such as mindfulness-based therapy (MBT), often comes primarily from small trials. Methods like funnel plots and other statistical methods that are used to assess the likelihood of publication bias, however, depend on having a sufficient number of large studies to anchor the analysis. When there is insufficient data to support statisticalmethods, an alternativemethod is needed. Capture-recapture is a method that has been used to estimate missing information and which has been used to estimate the number of articles that may have been missed by different search methods, for instance. Capture-recapture could also be used to compare completed trials identified via traditional database searches and clinical trial registries.

Objectives: To investigate (1) whether evidence syntheses of MBT effectiveness include non-published trial evidence and (2) whether risk of potential bias is adequately evaluated in MBT evidence reviews, and (3) to estimate the proportion of completed MBT trials that are not published.

Methods: PubMed, PsycInfo, CINAHL, Embase, ISI, SCOPUS, Cochrane were searched through 2012 for systematic reviews and meta-analyses on MBT. The same databases and clinical trial registries (ClinicalTrials.gov; Standard Randomized Controlled Trial Number Register; region-specific registries accessed through the World Health Organization registry search portal) were searched for trials of MBT targeting medical symptoms, including psychiatric symptoms.

Preliminary Results: Of > 20 published systematic reviews and meta-analyses, few assessed possible publication bias, though all who did concluded, using statistically-driven methods, that there was no publication bias. There are, however, a large number of unpublished trials, and we will estimate the number of these using the capture-recapture method.

Conclusions: The capture-recapture method may be a useful method to estimate possible publication bias in the context of relatively small trials.