Article type
Abstract
Background
The cost of producing a systematic review has been estimated to approximately $ 140,000. Using systematic reviews already produced is a way to effectively use resources, but it is paramount that the results of the reviews are reliable. Systematic reviews that have a high risk of bias have very limited use and are a waste of resources. The foundation of a systematic review is the literature search. When it is flawed there is a risk that relevant studies have been missed, making results unreliable.
Objectives
To investigate the proportion of published systematic reviews that have a high risk of bias, specifically when due to flaws in the design or reporting of the literature search strategy.
Methods
Risk of bias for all relevant systematic reviews identified in responses from the SBU Enquiry Service published on our website between 2021 and 2023, N=87, were extracted. It was also recorded whether the literature search strategy itself, or the reporting of it, was the reason for determining a systematic review as having a high risk of bias.
Results
For 7 of the responses assessment of risk of bias was not recorded, leaving a total of 80. Of those, 38 found no relevant systematic reviews. For the remaining 42 responses between 1 and 72 relevant systematic reviews were identified, adding up to a total amount of 587 systematic reviews. 480 of these, (82%) had a high risk of bias and 395 (67%) had problems in the design or reporting of literature searches. For systematic reviews published between 2020 and 2023, in total 221, the results are somewhat better, 173 (78%) had a high risk of bias and 136 (62%) had problems in the design or reporting of the literature search strategy.
Conclusion
In this study, an overwhelming majority of published systematic reviews have a high risk of bias, often due to flaws in the design or reporting of literature searches. This affects the reliability of the results, making resources dedicated to producing these reviews wasted.
The cost of producing a systematic review has been estimated to approximately $ 140,000. Using systematic reviews already produced is a way to effectively use resources, but it is paramount that the results of the reviews are reliable. Systematic reviews that have a high risk of bias have very limited use and are a waste of resources. The foundation of a systematic review is the literature search. When it is flawed there is a risk that relevant studies have been missed, making results unreliable.
Objectives
To investigate the proportion of published systematic reviews that have a high risk of bias, specifically when due to flaws in the design or reporting of the literature search strategy.
Methods
Risk of bias for all relevant systematic reviews identified in responses from the SBU Enquiry Service published on our website between 2021 and 2023, N=87, were extracted. It was also recorded whether the literature search strategy itself, or the reporting of it, was the reason for determining a systematic review as having a high risk of bias.
Results
For 7 of the responses assessment of risk of bias was not recorded, leaving a total of 80. Of those, 38 found no relevant systematic reviews. For the remaining 42 responses between 1 and 72 relevant systematic reviews were identified, adding up to a total amount of 587 systematic reviews. 480 of these, (82%) had a high risk of bias and 395 (67%) had problems in the design or reporting of literature searches. For systematic reviews published between 2020 and 2023, in total 221, the results are somewhat better, 173 (78%) had a high risk of bias and 136 (62%) had problems in the design or reporting of the literature search strategy.
Conclusion
In this study, an overwhelming majority of published systematic reviews have a high risk of bias, often due to flaws in the design or reporting of literature searches. This affects the reliability of the results, making resources dedicated to producing these reviews wasted.