Design, conduct, and analysis of safety among drug systematic reviews: a cross-sectional survey

Tags: Poster
Li L1, Deng K1, Busse J2, Zhou X3, Xu C1, Liu Z1, Ren Y1, Zou K1, Sun X1
1Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, 2Department of Health Research Methods, Evidence and Impact, McMaster University, 3Evidence-Based Medicine Research Center, School of Basic Science, Jiangxi University of Traditional Chinese Medicine

Background: well planned and rigorously conducted systematic reviews and meta-analyses provide the best evidence to inform drug safety, however, no surveys have specifically examined the methodological issues regarding drug safety.

Objectives: to examine, through a cross-sectional survey, the design, conduct, and analysis of safety among systematic reviews of drugs, and to explore factors associated with methodological quality.

Methods: we searched PubMed to identity systematic reviews published in the Cochrane Database of Systematic Reviews and 'Core Clinical Journals' indexed in 2015. We randomly sampled the studies - 1:1 of Cochrane and non-Cochrane reviews - assessing drug effects (including both efficacy and safety). We collected information regarding general characteristics, general methodological characteristics, and special characteristics of meta-analyses regarding the primary adverse outcome. We used the AMSTAR tool (Assessing the Methodological Quality of Systematic Reviews) to assess the methodological quality of included studies. We used multivariable linear regression analyses to explore the association of six prespecified variables with the AMSTAR scores.

Results: we included 120 systematic reviews; 60 Cochrane and 60 non-Cochrane reviews. The extent to which these systematic reviews and meta-analyses differed in design, conduct and analysis was substantial across items. Only 7.5% and 5.8% of reviews clearly defined safety outcomes and the primary safety outcome. Cochrane Reviews were better than non-Cochrane reviews at involving methodologists (P < 0.001), comprehensive databases searches (P = 0.001), literature screening (P < 0.001), duplicate data collection (P = 0.029), methods used to address reporting of missing participant data (P < 0.001), and using the GRADE approach to evaluate the quality of the evidence of safety outcome (P < 0.001). However, there were more non-Cochrane reviews conducting meta-analysis to pooled the primary adverse outcome (P = 0.003), and using funnel plot (P = 0.02) or Egger's test to test the publication bias of the primary adverse outcome (P = 0.049). The scores for the AMSTAR checklist were higher among Cochrane versus non-Cochrane systematic reviews (median 10.0 [first, third quartile: 9.0, 11.0] vs. 8.5 [8.5, 9.5]; P < 0.001). Multivariable linear regression analysis showed that Cochrane Reviews (adjusted β = 1.45, 95% CI 1.04 to 1.86; P < 0.001) were associated with better methodological quality, while systematic reviews assessing the effects of drugs in medical specialties were associated with poorer methodological quality (adjusted β = -0.47, 95% CI -0.87 to -0.06; P = 0.03).

Conclusions: Cochrane Systematic Reviews had better methodological quality than non-Cochrane reviews; the type of systematic review and area of disease were factors associated with methodological quality. There is room for improvement concerning the reported designs, conduct, and analysis of safety among drugs systematic reviews.

Patient or healthcare consumer involvement: none