An online implementation of the ROBINS-I (risk of bias in non-randomized studies – of interventions) tool

Tags: Oral
McGuinness LA1, Hutton D1, Sterne JAC1, Reeves BC1, Savovic J1, Higgins JPT1
1Bristol Medical School, University of Bristol

Background: Non-randomized studies of interventions (NRSI) can provide useful data on the effects of healthcare interventions but are likely to be subject to confounding and other biases, some of which do not apply to trials. The ROBINS-I (risk of bias in non-randomized studies – of interventions) tool provides a comprehensive assessment of risk of bias in NRSI, facilitating their inclusion in robust systematic reviews.

Objective: To describe the features of an online implementation of the ROBINS-I tool.

Methods: The online implementation contains all the elements of ROBINS-I (target trial specification, preliminary consideration of confounders/co-interventions, signalling questions informing 'Risk of bias' judgements within seven bias domains, derivation of an overall 'Risk of bias' judgement) and incorporates significant additional functionality. The implementation was built using ASP.net and Microsoft SQL Server.

Key features: Information necessary to perform the assessment, including recent updates to ROBINS-I signalling questions, question-specific guidance and relevant study documents (publications/protocols/etc.), is presented in a user-friendly layout. Additional features include:

- different levels of user access depending on prespecified roles (e.g. only review owners can create, edit and assign assessments to other users);

- signalling questions asked only when relevant (conditional on previous answers);

- algorithms for domain-level risk of bias judgements based on answers to signalling questions, with the potential for manual override;

- comparison function, allowing rapid reconciliation of assessments made by different raters;

- online help.

Conclusions: Online implementation of ROBINS-I will aid both novice and experienced reviewers by making the tool easier and quicker to use and by increasing the accuracy of their assessments. This work will pave the way for similar implementations of other tools, including the Risk of Bias 2.0 tool for trials and the forthcoming ROBINS-E (risk of bias in non-randomized studies – of exposures).

Patient/consumer involvement: Risk of bias assessment requires advanced epidemiological knowledge, so involving patients in its development is challenging. However, we involved the target ‘consumers’ for this online implementation, review authors, during development to ensure the tool was fit for use.