Article type
Year
Abstract
Background: Studies examining selective reporting of drug trial data show that positive results are more likely to be published than null or negative results. Consequently, meta-analyses based only on published data may over-estimate drug efficacy and under-estimate drug harms. Food and Drug Administration (FDA) reviews of drug trials are publicly available, making them a good source of unpublished trial data for inclusion into meta-analyses.
Objective: This study examines the effect of including unpublished FDA trial data on the results of published meta-analyses across a variety of drug classes.
Methods: We studied 24 drugs approved by the FDA between 2001-2002 with previously identified unpublished outcome data. We performed a systematic search of PubMed, Embase, and the Cochrane library in November 2010 to identify relevant meta-analyses for each of the study drugs. Two authors independently screened meta-analyses and selected one for analysis when multiple meta-analyses for a drug were included. Two authors independently extracted data from meta-analyses and the FDA reviews. We calculated summary statistics both with and without unpublished trial data.
Results: After excluding 11 drugs in which no relevant meta-analyses were identified, 13 drugs and 42 meta-analyses met our inclusion criteria. Each drug had 1 to 8 corresponding meta-analyses (median = 2). To date, we have completed analyses for 4drugs and recalculated summary statistics for 15outcomes. For 11outcomes, the change in summary statistics after inclusion of unpublished data demonstrated less benefit or more harm of the drug (range in percent change of summary statistics: 4% - 41%). Four outcomes showed more benefit of the drug after inclusion of unpublished data (range in percent change of summary statistics: 4% to 9%). Discussion: This preliminary analysis suggests that the inclusion of unpublished trial data in meta-analyses affects their results. The magnitude and direction of the effect varies between drugs.
Objective: This study examines the effect of including unpublished FDA trial data on the results of published meta-analyses across a variety of drug classes.
Methods: We studied 24 drugs approved by the FDA between 2001-2002 with previously identified unpublished outcome data. We performed a systematic search of PubMed, Embase, and the Cochrane library in November 2010 to identify relevant meta-analyses for each of the study drugs. Two authors independently screened meta-analyses and selected one for analysis when multiple meta-analyses for a drug were included. Two authors independently extracted data from meta-analyses and the FDA reviews. We calculated summary statistics both with and without unpublished trial data.
Results: After excluding 11 drugs in which no relevant meta-analyses were identified, 13 drugs and 42 meta-analyses met our inclusion criteria. Each drug had 1 to 8 corresponding meta-analyses (median = 2). To date, we have completed analyses for 4drugs and recalculated summary statistics for 15outcomes. For 11outcomes, the change in summary statistics after inclusion of unpublished data demonstrated less benefit or more harm of the drug (range in percent change of summary statistics: 4% - 41%). Four outcomes showed more benefit of the drug after inclusion of unpublished data (range in percent change of summary statistics: 4% to 9%). Discussion: This preliminary analysis suggests that the inclusion of unpublished trial data in meta-analyses affects their results. The magnitude and direction of the effect varies between drugs.