Article type
Year
Abstract
Background: Recently, A Cochrane Risk Of Bias Assessment Tool (ACROBAT) was released to examine internal validity (risk of bias) in non-randomized studies of interventions (NRSI). The applicability of ACROBA- NRSI to studies dealing with exposures or interventions in environmental health has not yet been explored.
Objectives: This study evaluated the application of ACROBAT-NRSI in studies of environmental health exposure.
Methods: Two independent reviewers sequentially applied ACROBAT-NRSI to two systematic reviews containing 14 and 17 individual studies of environmental exposures. Material available to reviewers included instructions for application of the ACROBAT-NRSI, including a list of possible confounders specific to the exposures considered in the two reviews. After completing the first and second assessments with the tool, reviewers provided feedback regarding clarity of the instrument and ease of application, highlighting barriers to use. Modifications to enhance usability for studies of exposures (rather than interventions) were made to the tool.
Results: Based on feedback from reviewers, two modifications were required:
1. replacement of 'intervention' with 'exposure' throughout the document; and
2. provision of additional explanation and examples for application of the tool to studies using a cross-sectional design.
Modification to the tool increased reviewers’ understanding during the second application; however, disagreements occurred when judgments on domain-level risk of bias were made, specifically in domains assessing bias due to confounding and bias in the selection of the reported result.
Conclusion: Modifications to the tool were limited and improved understanding, as well as reduced disagreement, among reviewers. We identified areas requiring more clarification to further improve consistency of the judgments. We will perform additional work examining studies of exposure with the goal of providing a definitive instrument to assess risk of bias in studies of environmental exposures.
Objectives: This study evaluated the application of ACROBAT-NRSI in studies of environmental health exposure.
Methods: Two independent reviewers sequentially applied ACROBAT-NRSI to two systematic reviews containing 14 and 17 individual studies of environmental exposures. Material available to reviewers included instructions for application of the ACROBAT-NRSI, including a list of possible confounders specific to the exposures considered in the two reviews. After completing the first and second assessments with the tool, reviewers provided feedback regarding clarity of the instrument and ease of application, highlighting barriers to use. Modifications to enhance usability for studies of exposures (rather than interventions) were made to the tool.
Results: Based on feedback from reviewers, two modifications were required:
1. replacement of 'intervention' with 'exposure' throughout the document; and
2. provision of additional explanation and examples for application of the tool to studies using a cross-sectional design.
Modification to the tool increased reviewers’ understanding during the second application; however, disagreements occurred when judgments on domain-level risk of bias were made, specifically in domains assessing bias due to confounding and bias in the selection of the reported result.
Conclusion: Modifications to the tool were limited and improved understanding, as well as reduced disagreement, among reviewers. We identified areas requiring more clarification to further improve consistency of the judgments. We will perform additional work examining studies of exposure with the goal of providing a definitive instrument to assess risk of bias in studies of environmental exposures.