Article type
Year
Abstract
Background:
Systematic reviews are now generally consulted as the primary source of evidence for the effects of an intervention. As with any study, systematic flaws or limitations in the design or conduct of the review have the potential to bias results. Several tools are available for the critical appraisal and quality assessment of a systematic review. However, none aims specifically to assess the risk of bias introduced by the review conduct or its interpretation of findings.
Objectives:
To describe the new 'ROBIS' tool, developed using rigorous methodology, to assess the risk of bias in a systematic review.
Methods:
We used a five phased approach to develop 'ROBIS': definition of the scope of the tool, development of the evidence base concerning biases in systematic reviews, face to face meeting, the Delphi prediction method to develop a first version of the tool, and refinement through piloting. We established a broad team including methodological experts in systematic reviews and review authors.
Results:
We agreed that 'ROBIS' should consider the relevance of a review to the research question at hand (where appropriate) and assess the risk of bias in the review. Discussions at the face-to-face meeting informed decisions and the subsequent Delphi procedure led to a tool consisting of five key domains: (1) study eligibility criteria, (2) identification and selection of studies, (3) data collection and study appraisal, (4) synthesis and findings, and (5) interpretation. A series of questions within each domain elicits information about possible limitations of the systematic review, leading to a judgement about the risk of bias within that domain (low, high or unclear risk of bias). Piloting of 'ROBIS' is ongoing at the time of writing.
Conclusions:
The development of 'ROBIS' was evidence based and included detailed piloting and refinement. We believe that 'ROBIS' will allow objective rating of the risk of bias and relevance of systematic reviews.
Systematic reviews are now generally consulted as the primary source of evidence for the effects of an intervention. As with any study, systematic flaws or limitations in the design or conduct of the review have the potential to bias results. Several tools are available for the critical appraisal and quality assessment of a systematic review. However, none aims specifically to assess the risk of bias introduced by the review conduct or its interpretation of findings.
Objectives:
To describe the new 'ROBIS' tool, developed using rigorous methodology, to assess the risk of bias in a systematic review.
Methods:
We used a five phased approach to develop 'ROBIS': definition of the scope of the tool, development of the evidence base concerning biases in systematic reviews, face to face meeting, the Delphi prediction method to develop a first version of the tool, and refinement through piloting. We established a broad team including methodological experts in systematic reviews and review authors.
Results:
We agreed that 'ROBIS' should consider the relevance of a review to the research question at hand (where appropriate) and assess the risk of bias in the review. Discussions at the face-to-face meeting informed decisions and the subsequent Delphi procedure led to a tool consisting of five key domains: (1) study eligibility criteria, (2) identification and selection of studies, (3) data collection and study appraisal, (4) synthesis and findings, and (5) interpretation. A series of questions within each domain elicits information about possible limitations of the systematic review, leading to a judgement about the risk of bias within that domain (low, high or unclear risk of bias). Piloting of 'ROBIS' is ongoing at the time of writing.
Conclusions:
The development of 'ROBIS' was evidence based and included detailed piloting and refinement. We believe that 'ROBIS' will allow objective rating of the risk of bias and relevance of systematic reviews.