Article type
Abstract
Background: A substantial number of systematic reviews (SR) are failing to follow recommendations for the conduct and reporting of search methods despite the availability of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and the Cochrane Handbook for Systematic Reviews of Interventions.
Objectives: We aimed to analyse the main reasons for SRs being assessed as high risk of bias in Domain 2 (Identification and Selection of Studies) of the ROBIS Risk-of-Bias tool.
Methods: The KSR Evidence database has over 30 000 SRs critically appraised according to the ROBIS tool. These were filtered to those assessed to be of high risk of bias in Domain 2 and, from this set, a random sample was selected for further analysis.
Results: From our piloted sample we found the most frequent reasons for SRs receiving a high risk of bias assessment in Domain 2 was the failure to search for non-English publications; and no undertaking of supplementary searches additional to healthcare database searching, i.e. no hand searching, grey literature or citation searching. A substantial number of SRs failed to report search strategies and/or the study selection process adequately. Of the SRs assessed as high risk of bias in Domain 2, 98 per cent are ultimately assessed as having a high risk of bias overall for the full review.
Conclusion: SRs are being assessed as of high risk of bias in Domain 2 for reasons which, in some cases, could easily be avoided. Improved reporting and omission of language limits would add little extra work but could improve the academic rigour of and increase the value of the research undertaken. As SRs are expected to be transparent and reproducible, we believe common failures in this domain undermine the overall value of SRs and, in so doing, contribute to unnecessary research waste.
Objectives: We aimed to analyse the main reasons for SRs being assessed as high risk of bias in Domain 2 (Identification and Selection of Studies) of the ROBIS Risk-of-Bias tool.
Methods: The KSR Evidence database has over 30 000 SRs critically appraised according to the ROBIS tool. These were filtered to those assessed to be of high risk of bias in Domain 2 and, from this set, a random sample was selected for further analysis.
Results: From our piloted sample we found the most frequent reasons for SRs receiving a high risk of bias assessment in Domain 2 was the failure to search for non-English publications; and no undertaking of supplementary searches additional to healthcare database searching, i.e. no hand searching, grey literature or citation searching. A substantial number of SRs failed to report search strategies and/or the study selection process adequately. Of the SRs assessed as high risk of bias in Domain 2, 98 per cent are ultimately assessed as having a high risk of bias overall for the full review.
Conclusion: SRs are being assessed as of high risk of bias in Domain 2 for reasons which, in some cases, could easily be avoided. Improved reporting and omission of language limits would add little extra work but could improve the academic rigour of and increase the value of the research undertaken. As SRs are expected to be transparent and reproducible, we believe common failures in this domain undermine the overall value of SRs and, in so doing, contribute to unnecessary research waste.