Article type
Year
Abstract
Background: Combining biased evidence propagates biased systematic review findings in the absence of an appropriately conducted 'Risk of bias' (RoB) assessment. Systematic reviews including research studies at high or unclear RoB yield inflated effect estimates compared to reviews possessing studies at low RoB. The type of tool selected and evaluation method used to undertake a RoB assessment are essential for a true interpretation of review findings.
Objectives: To determine RoB assessment prevalence, tool types, and evaluation methods across systematic reviews in a leading international sports and exercise medicine journal.
Methods: A systematic search for review articles in the British Journal of Sports Medicine archive identified and included systematic reviews published from 1 January 2016 to 10 May 2017 (Figure 1). Independently, two authors screened review titles and abstracts and, following agreement in this process, abstracted descriptive information related to each systematic review manually, using a predetermined data extraction template.
The primary variables of interest included:
1) prevalence of RoB assessments used across the included systematic reviews;
2) type of RoB tool used;
3) type of RoB assessment performed (e.g. domain-level, outcome-level, etc.).
Results: We included 66 systematic reviews, 65 (98.5%) of which undertook a critical appraisal of included studies. Only 11 (16.9%) implemented a RoB tool that evaluated the internal validity of included studies. To assess RoB, the remaining systematic reviews implemented tools that evaluated methodological quality (47.7%), reporting (1.5%), a combination of internal validity, external validity, methodological quality, and reporting (18.5%), or 'other' (15.4%). Twelve (18.5%) reviews performed an outcome-level RoB assessment. Only three (4.6%) systematic reviews undertook an outcome-level RoB assessment in the context of bias-specific domains.
Conclusions: RoB assessments are commonly used across systematic reviews in a leading international sports and exercise medicine journal. However, suboptimal tools and assessment methods, which evaluate domains extraneous to internal validity, are frequently implemented to perform RoB assessments. This substantially limits the extent to which readers can interpret the effect of bias on systematic review findings.
Objectives: To determine RoB assessment prevalence, tool types, and evaluation methods across systematic reviews in a leading international sports and exercise medicine journal.
Methods: A systematic search for review articles in the British Journal of Sports Medicine archive identified and included systematic reviews published from 1 January 2016 to 10 May 2017 (Figure 1). Independently, two authors screened review titles and abstracts and, following agreement in this process, abstracted descriptive information related to each systematic review manually, using a predetermined data extraction template.
The primary variables of interest included:
1) prevalence of RoB assessments used across the included systematic reviews;
2) type of RoB tool used;
3) type of RoB assessment performed (e.g. domain-level, outcome-level, etc.).
Results: We included 66 systematic reviews, 65 (98.5%) of which undertook a critical appraisal of included studies. Only 11 (16.9%) implemented a RoB tool that evaluated the internal validity of included studies. To assess RoB, the remaining systematic reviews implemented tools that evaluated methodological quality (47.7%), reporting (1.5%), a combination of internal validity, external validity, methodological quality, and reporting (18.5%), or 'other' (15.4%). Twelve (18.5%) reviews performed an outcome-level RoB assessment. Only three (4.6%) systematic reviews undertook an outcome-level RoB assessment in the context of bias-specific domains.
Conclusions: RoB assessments are commonly used across systematic reviews in a leading international sports and exercise medicine journal. However, suboptimal tools and assessment methods, which evaluate domains extraneous to internal validity, are frequently implemented to perform RoB assessments. This substantially limits the extent to which readers can interpret the effect of bias on systematic review findings.