Does applicability make the difference? Two systematic reviews on the same topic

Angelescu K1, Nussbaumer B2, Sieben W1, Scheibler F1, Sauerland S1
1Institute for Quality and Efficiency in Health Care (IQWiG), Germany, 2Department for Evidence-Based Medicine and Clinical Epidemiology, Danube University Krems, Austria

Background: Most guidelines recommend screening for and treatment of asymptomatic bacteriuria (ASB) in pregnancy. A Cochrane Review (CR) on antibiotic treatment of ASB in pregnancy provides the basis of this recommendation. However, an Institute for Quality and Efficiency in Health Care (IQWiG) report on benefits and harms of screening for ASB concluded that the benefit is not proven. Objective: We compared the two systematic reviews to find out why they have come to differing conclusions. Methods: We compared PICO, study pool, risk of bias (RoB) assessment, effect estimates, assessment of applicability, and conclusions. Results: Differences in inclusion and exclusion criteria result in different study pools. The IQWiG report included only 3 of 14 studies that were included in the CR. However, effect estimates of pyelonephritis rates are roughly comparable (CR: relative risk = 0.23, 95% confidence interval (CI) 0.13 to 0.41; IQWiG: odds ratio = 0.21, 95% CI 0.07 to 0.59). Studies date back from the 1960s to the 1980s. Both reviews rate the RoB of included studies as high. Both do not describe a methodology to assess applicability. However, in the CR applicability is discussed with respect to study medication and microbiologic methods, but not generally rejected. The IQWiG report regards interventions in 2 of 3 included studies as not applicable to current health care. In conclusion, the CR identifies preventive effects, but recommends interpreting the results cautiously. The IQWiG report, which, unlike the CR, comprises a dichotomised statement on benefit and harms, regards the benefit of an ASB treatment as not proven due to serious concerns regarding applicability. Conclusion: Despite differences in detail, overall conclusions regarding the effect of treatment do not differ much. Both works see the need of further randomised trials. The main difference is seen in the significance of applicability and derived conclusions. To date there is no well-developed methodology for assessing applicability. However, different domains of applicability (population, setting, interventions, outcomes, follow-up) should be assessed as recommended in current EUnetHTA guidance.