The agreement of treatment-effect estimates from rapid reviews using abbreviated literature searches and traditional Cochrane reviews: A meta-epidemiological study

Tags: Poster
Ladanie A1, Klerings I2, Wagner G2, Heise TL3, Dobrescu AI4, Armijo-Olivo S5, Stratil JM6, Lhachimi SK3, van Noord M7, Mittermayr T8, Gartlehner G9, Nussbaumer-Streit B10, Hemkens LG11
1Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel & Swiss Tropical and Public Health Institute, University of Basel, Basel, 2Department of Evidence-based Medicine and Clinical Epidemiology, Krems, 3Leibniz Institute for Prevention Research and Epidemiology – BIPS, University Bremen, 4Genetics Department, Victor Babes University of Medicine and Pharmacy Timisoara, 5Faculty of Rehabilitation Medicine, University of Alberta & Institute of Health Economics, Edmonton, Alberta, 6Institute for Medical Informatics, Biometry and Epidemiology, Ludwig-Maximilians-University, Munich, 7Medical Center Library & Archives, School of Medicine, Duke University, 8Ludwig Boltzmann Institute for Health Technology Assessment, 9Cochrane Austria, Danube University Krems, Krems & RTI International, NC, 10Cochrane Austria, Danube University Krems, Krems, 11Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel

Background: Rapid reviews have become a pragmatic alternative to traditional systematic reviews by streamlining methodological processes such as literature searches, aiming for faster provision of decision-relevant information. The agreement of effect estimates obtained from abbreviated literature searches as conducted in rapid reviews and those obtained from extensive searches as performed in Cochrane or Campbell reviews is unknown.

Objectives: To assess differences between treatment effects estimated from abbreviated and extensive literature searches.

Methods: We selected 44 Cochrane reviews published between 2012 and 2016 on diverse topics that had a binary outcome in the main summary of findings table (a subset of randomly selected reviews used for a related project [1]). We constructed 14 variants of abbreviated searches based on the original CSRs search strategy by combining searches in MEDLINE, Embase, or CENTRAL with or without searching of reference lists. From the first binary outcome’s meta-analysis of each CSR, we extracted trial results, recorded for each trial which abbreviated search variant identified it, and calculated a summary treatment effect estimate per search variant (summary odds ratio, sOR) including only trials identified by that variant using random-effects meta-analyses. We then determined how often sORs of abbreviated and original searches differed in the direction, in the inclusion of the null (i.e. nominal statistical significance), and beyond chance (i.e. the ratio of sORs excluded the null). We follow an epidemiological approach to evaluate the overall relationship of treatment effect sizes derived by abbreviated and original searches in an overarching meta-analysis across all 44 ratio of sORs.

Results: Analyses are ongoing and results will be available at time of the Summit.

Conclusions: Our results will systematically quantify the impact of faster and abbreviated searching on treatment effect estimates across a wide range of CSR topics.

[1] Nussbaumer-Streit et al. Assessing the validity of abbreviated literature searches for rapid reviews: protocol of a non-inferiority and meta-epidemiologic study. Sys rev. 2016;5:197