Validation of the Spanish version of the Risk of Bias in Systematic Reviews (ROBIS) tool

Franco J1, Simancas-Racines D2, Nuñez S3, Delgado-Ron A3, Loézar Hernández C4, Vargas Peirano M4, Pérez Bracchiglione J4, Papuzinski C4, Madrid E4, Bravo G5, Whiting P6, Savović J6, Churchill R7
1Instituto Universitario Hospital Italiano, Associate Cochrane Centre, 2Centre for Reseach in Public Health and Clinical Epidemiology, School of Health Sciences Eugenio Espejo, Universidad Tecnológica Equinoccial, Ecuadorian Associate Cochrane Center, 3Centre for Reseach in Public Health and Clinical Epidemiology, School of Health Sciences Eugenio Espejo. Universidad Tecnológica Equinoccial, Ecuadorian Associate Cochrane Center, 4Interdisciplinary Centre for Health Studies, University of Valparaíso, Associate Cochrane Centre, 5Epistemonikos Foundation / Pontifical Catholic University of Chile, de Santiago, 6University of Bristol, 7University of York

Background:

Systematic reviews can be a reliable source for information-based decision-making. They should retrieve all relevant primary studies addressing a research question, use robust methods for appraising the quality of included studies, and apply appropriate and transparent methods to synthesise these. The Risk of Bias in Systematic Reviews (ROBIS) tool was designed to identify the risk of bias in the review process (sometimes called 'metabias') in order to guide users, guideline developers and other stakeholders in assessing the reliability of systematic review conclusions. This tool has previously only been available in English; there is a growing need to assess the methods of systematic reviews in Spanish.

Objectives:

To assess the validity and usability of the Spanish version of the ROBIS tool.

Methods:

1) Development

We performed two independent translations of the ROBIS tool into Spanish. We established a group of experts in evidence-based medicine (EBM) who reviewed the two versions and drafted a preliminary version assessing face validity. We then pilot tested the tool with six systematic reviews using both the original tool and the translated tool, in order to identify problems in its effective implementation, which we corrected.

2) Validation

We plan to apply the original ROBIS tool using a group of methodologists and trainees in EBM who will independently assess systematic reviews in pairs using both the original tool and the translated tool during EBM meetings, including the Cochrane Iberoamerican Meeting in Cuba (May 2018). We plan to evaluate the agreement between the assessments of each domain comparing the original tool and the translated tool. Additionally, we will record the time taken to apply it, and we will collect feedback from the participants regarding its usability.

Results and conclusions:

We have developed a Spanish version of the ROBIS tool that has received positive feedback during our initial pilot testing. We expect to refine this tool throughout this year and we will present the results of this experience at the Colloquium. We believe that this refined version will help the formal assessment of metabias in systematic reviews in Spanish and the development of overviews.

Patient or healthcare consumer involvement:

Our team is formed of methodologists and EBM experts of which five, including the main author, are practising clinicians.