Is further research really needed? Evidence from published comparative effectiveness reviews

Nikolakopoulou A1, Mavridis D1, Furukawa T2, Cipriani A3, Egger M4, Salanti G4
1University of Ioannina, School of Medicine, Greece, 2Kyoto University, Department of Clinical Epidemiology, Japan, 3University of Oxford, Department of Psychiatry, UK, 4Institute of Social and Preventive Medicine (ISPM), University of Bern, Switzerland

Background: Cochrane is moving towards prioritizing updates of systematic reviews based on the needs of healthcare and health policy decision makers. Despite the increasing interest in determining when and how to update systematic reviews, there is still no consensus on the appropriate methodology to be applied. Formal sequential methods have been developed for pairwise and network meta-analysis (NMA) to adjust for the type I error, which is inflated in a continuously updated, prospective cumulative meta-analysis.

Objectives: We aim to examine to what extent published reviews provide strong evidence and firm conclusions upon treatment comparisons of interest and whether NMA achieves conclusiveness earlier than pairwise meta-analysis.

Methods: We applied recently developed sequential NMA methodology to 30 recently published NMAs that compared pharmacological or surgical interventions. In each network, we focussed on a single treatment comparison, the choice of which was based on their importance with respect to guideline development and conclusions from official bodies. We evaluated the conclusiveness of the selected comparisons in the included networks using both direct and NMA sequential methodology and considered stopping both for efficacy and futility.

Results: Most systematic reviews are inconclusive for the treatment comparison of interest, using either pairwise meta-analysis or NMA; such a situation is illustrated in panel A of Fig 1. Network effects yield more precise results and in certain cases, for instance in panel B of Fig 1, formal decisions of stopping would have been made using NMA, while direct evidence would remain inconclusive. The number of cases of conclusiveness achieved using indirect evidence and the hazard ratio for conclusiveness between direct and network evidence will be presented.

Conclusions: Wasted research can be reduced significantly with the adoption of living cumulative NMAs, updated as new research becomes available. The use of sequential methods in such reviews may contribute to preventing the allocation of participants to treatments that have proved to be inferior.