Assessing the validity of an ultra-rapid review strategy: four case studies from oncology and public health

Article type
Authors
Affengruber L1, Wagner G1, Waffenschmidt S2, Nussbaumer-Streit B1, Thaler K3, Lhachimi S4, Patel S5, Gartlehner G6
1Department of Evidence-based Medicine and Clinical Epidemiology, Danube University Krems, Krems & Cochrane Austria, Danube University Krems, Krems
2Information Management Unit, Institute for Quality and Efficiency in Health Care, Cologne
3Medical Department I, Hanusch Krankenhaus der Wiener Gebietskrankenkasse, Vienna
4Research Group Evidence-Based Public Health, Leibniz Institute for Epidemiology and Prevention Research (BIPS), Bremen & Health Sciences Bremen, Institute for Public Health and Nursing, University of Bremen, Bremen
5RTI-University of North Carolina Evidence-based Practice Center, RTI International, North Carolina & Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, North Carolina
6Department of Evidence-based Medicine and Clinical Epidemiology, Danube University Krems, Krems; Cochrane Austria, Danube University Krems, Krems & RTI International, North Carolina
Abstract
Background: Rapid reviews have become a pragmatic alternative to systematic reviews (SR) because they meet time-sensitive needs of decision-makers. By simplifying methodological steps, results from rapid evidence syntheses are available within a few weeks or months. Ultra-rapid reviews (URR) employ multiple methodological shortcuts and are conducted in a very limited timeframe (a few days). Because of these shortcuts, URR have the potential to be less reliable than SR.

Objectives: The primary objective of this study was to assess whether an URR approach leads to different overall conclusions about treatment effects compared to a SR approach. This study also sought to explore whether practicality and validity of the URR strategy vary between clinical and public health topics.

Methods: We used a non-random (convenience) sample of four ongoing Cochrane Reviews as reference standards against which we compared the validity of the URR strategy with strongly abbreviated literature searches and single screening of abstracts and full texts. Two reviews addressed oncological topics and two were on public health topics.

For each topic we conducted a simple-structured Boolean search in combination with the first 20 similar articles in PubMed. The starter sets for the similar articles function in PubMed were highly relevant, published studies provided by the Cochrane review authors. Literature searches were conducted centrally by an information specialist. For each of the four topics, three reviewers screened literature independently.

We will survey Cochrane review authors to solicit whether the detected bodies of evidence from the URR would lead to different conclusions than the respective body of evidence detected by the Cochrane review. We will assess whether effect estimates differ between URR and SR.

Results: Final results will be available by the time of the Cochrane Colloquium.

Conclusions: Findings of this project will provide information about the practicality and validity of using an URR approach.

Patient or healthcare consumer involvement: Usually rapid reviews engage patients and healthcare providers in the review process to focus on patient relevant health outcomes. The synthesized evidence is then presented simply and concisely for informed decision-making.