Frequency of use and reporting adequacy of Cochrane RoB 2 tool in non-Cochrane systematic reviews published in 2020: meta-research study

Oral
2023 London
Babic A1, Barcot O2, Viskovic T1, Saric F3, Kirkovski A4, Barun I5, Krizanac Z2, Ananda R6, Fuentes Barreiro YV7, Malih N8, Dimcea DA9, Ordulj J10, Weerasekara I11, Spezia M12, Zuljevic MF13, Suto J14, Tancredi L15, Pijuk A16, Sammali S17, Iascone V18, von Groote T19, Poklepovic Pericic T20, Puljak L21
1Institute of Emergency Medicine in Split-Dalmatia County, Split, Croatia
2Department of Surgery, University Hospital Split, Split, Croatia
3Department of Radiology, University Hospital Split, Split Croatia
4PZU MK & RR Centar Medikal, Bitola, Macedonia
5Department of Ophthalmology, University Hospital Split, Split, Croatia
6Department of General Medicine, Box Hill Hospital, Eastern Health, Box Hill, Australia
7Faculty of medicine, Universidad de La Sabana, Bogota, Colombia
8Research Group on Global Health and Human Development, University of the Balearic Islands (UIB), Palma, Spain
9Department of Obstetrics- Gynaecology, Elias Emergency University Hospital, Bucharest, Romania
10Dental clinic, Dugo Selo, Croatia
11Department of Health and Functioning, Faculty of Health and Social Sciences, Western Norway University of Applied Sciences, Bergen, Norway
12University of Padua, Padua, Italy
13Centre for Evidence-based Medicine, University of Split School of Medicine, Split, Croatia
14Department of oncology and radiotherapy, Clinical Hospital Centre Split, Croatia
15Geriatric rehabilitation clinic of the Hessing Foundation, Augsburg, Germany; Medical School, Coburg, Germany
16Division of Hematology, Department of Internal Medicine, University Hospital of Split, Split, Croatia
17University of Bologna, Bologna, Italy; University of Florence, Florence, Italy
18University of Bologna, Bologna, Italy
19Department of Anaesthesiology, Intensive Care and Pain Medicine, University Hospital Münster, Münster, Germany
20Department of Research in Biomedicine and Health, University of Split School of Medicine, Split, Croatia
21Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Zagreb, Croatia
Background: Risk of bias (RoB) assessment is an integral part of the systematic review methodology. New version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019. It has been reported that the Cochrane RoB tool from 2011 was used inadequately in Cochrane and non-Cochrane reviews. There are insufficient data about the use of RoB 2 tool.
Objectives: This study aimed to analyze the frequency of usage of RoB 2 tool and the adequacy of reporting RoB 2 assessments in non-Cochrane reviews published in 2020.
Methods: We conducted a meta-research study. We included non-Cochrane systematic reviews of interventions published in 2020. For the reviews that have used RoB 2 tool, we analyzed how the findings of the RoB 2 assessment were reported.
Results: Among 3880 analyzed reviews, the authors most frequently reported use of the Cochrane 2011 RoB tool (N=2228; 57.4%), followed by Cochrane RoB 2 tool (N=267; 6.9%). From 267 reviews that reported using RoB 2 tool, 213 (79.8%) actually used RoB 2 tool. Among 213 reviews that used RoB 2 tool, 180 (85.4%) used all domains specified by the RoB 2 tool.
In 26 (12.2%) reviews, erroneous statements were used to indicate RoB 2 assessment. Only 20 (9.4%) reviews presented a complete RoB 2 assessment through a detailed table with answers to all signaling questions. The judgment of risk of bias through RoB 2 tool was not justified by a comment in 158 (74.2%) of reviews. Only in 33 (14.5%) of reviews, the judgment in all domains was justified in the accompanying comment.
Conclusion: The majority of non-Cochrane reviews published in 2020 still used RoB 2011 tool and many authors reported the results of RoB 2 assessment inadequately. Interventions for improving RoB 2 tool use in non-Cochrane reviews are warranted.
Relevance and importance to patients: Efforts to improve research methodology and how studies are conducted can lead to better conducted and written studies, and better evidence that can be used for managing patients.