A comparison of statistical methods for identifying out-ofdate systematic reviews

Tags: Poster
Pattanittum P1, Laopaiboon M1, Moher D2, Lumbiganon P3, Ngamjarus C1
1Thai Cochrane Network, Thailand, and Department of Biostatistics and Demography, Faculty of Public Health, Khon Kaen University, Khon Kaen, Thailand, 2Clinical Epidemiology Program, Ottawa Hospital Research Institute, AND Department of Epidemiology and Community Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada, 3Thai Cochrane Network, Thailand, AND Department of Obstetrics and Gynaecology, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand

Background: Systematic reviews (SRs) can provide accurate and reliable evidence, typically about the effectiveness of health interventions. Evidence is dynamic, and if SRs are out-of-date this information may not be useful; it may even be harmful.

Objectives: To compare five statistical methods in terms of agreement and practical in identifying out-of-date SR.

Methods: A retrospective cohort of SRs registered in the Cochrane Pregnancy and Childbirth Group (CPCG), published between 2008 and 2010, were considered for inclusion. For each eligible CPCG review, data were extracted and ‘3-years previous’ meta-analyses were assessed for the need to update, given the data from the most recent 3 years. Each of the five statistical methods were used, with random effects analyses throughout the study.

Results: Eighty reviews were included in this study; most were in the area of induction of labour. The numbers of reviews identified as being out-of-date using the Ottawa, recursive cumulative meta-analysis (CMA), and Barrowman methods were 34, 7, and 7 respectively. The overall agreement among the three discriminating statistical methods was slight (Kappa = 0.14; 95% CI 0.05–0.23). The recursive CMA method was found to be a practical in terms of data requirements and ease of calculation. No reviews were identified as being out-of-date using the simulation-based power method, or the CMA for sufficiency and stability method.

Conclusions: The recursive cumulative meta-analysis method should be a practical method to examine the need to update SRs.