Article type
Year
Abstract
Objective
The aim of this research project was to investigate the risk of bias (RoB) in systematic reviews (SRs) conducted in rare diseases using a random sample of SRs (and their corresponding quality appraisals using the ROBIS tool) and assess their compliance with PRISMA reporting standards.
Methods
Two random samples of SR publications from 2016 and 2017 were identified from the KSR Evidence database of systematic reviews (ksrevidence.com). These were screened for systematic reviews of rare diseases. Rare disease status was confirmed using ORPHANET (a unique database for rare diseases and orphan drugs). Data were extracted on review type, study characteristics, ROBIS quality assessment and PRISMA compliance.
Results
From 1026 SRs, 25 (2.4%) were identified that reported on rare diseases. These included: systemic sclerosis, cystic fibrosis, Guillain-Barré syndrome, myasthenia gravis, Kawasaki disease, Churg Strauss syndrome, Maroteaux-Lamy syndrome and rare cancers such as Burkitt’s lymphoma, sarcoidosis and non-Hodgkin lymphoma. Of the 25 SRs, ten (40%) were interventional studies, two (8%) diagnostic, two (8%) epidemiological and 11 (44%) investigated other or multiple research questions. Two (of 25) SRs (8%) were at low risk of bias, two (8%) unclear risk of bias and 21 (84%) high risk of bias. The main areas of concern were not reporting search strategies, language limitations and no or inappropriate RoB assessment of included studies. Forty percent of studies stated that they were PRISMA complaint; however, 80% of these did not report a search strategy and 40% did not assess RoB.
Conclusion
Systematic reviews of rare diseases represented approximately 2% of all reviews. Most studies were at high RoB. Given the paucity of research in this area, it is important to encourage good quality research and highlight the areas of concern. There are five relatively simple ways to potentially improve the risk of bias of rare disease systematic reviews: 1) report a full search strategy; 2) do not apply language restrictions; 3) include conference abstracts; 4) use an appropriate RoB tool; and 5) avoid inappropriate pooling. It is not sufficient to state a study is PRISMA compliant; compliance must be demonstrated.
Patient or healthcare consumer involvement:
None
The aim of this research project was to investigate the risk of bias (RoB) in systematic reviews (SRs) conducted in rare diseases using a random sample of SRs (and their corresponding quality appraisals using the ROBIS tool) and assess their compliance with PRISMA reporting standards.
Methods
Two random samples of SR publications from 2016 and 2017 were identified from the KSR Evidence database of systematic reviews (ksrevidence.com). These were screened for systematic reviews of rare diseases. Rare disease status was confirmed using ORPHANET (a unique database for rare diseases and orphan drugs). Data were extracted on review type, study characteristics, ROBIS quality assessment and PRISMA compliance.
Results
From 1026 SRs, 25 (2.4%) were identified that reported on rare diseases. These included: systemic sclerosis, cystic fibrosis, Guillain-Barré syndrome, myasthenia gravis, Kawasaki disease, Churg Strauss syndrome, Maroteaux-Lamy syndrome and rare cancers such as Burkitt’s lymphoma, sarcoidosis and non-Hodgkin lymphoma. Of the 25 SRs, ten (40%) were interventional studies, two (8%) diagnostic, two (8%) epidemiological and 11 (44%) investigated other or multiple research questions. Two (of 25) SRs (8%) were at low risk of bias, two (8%) unclear risk of bias and 21 (84%) high risk of bias. The main areas of concern were not reporting search strategies, language limitations and no or inappropriate RoB assessment of included studies. Forty percent of studies stated that they were PRISMA complaint; however, 80% of these did not report a search strategy and 40% did not assess RoB.
Conclusion
Systematic reviews of rare diseases represented approximately 2% of all reviews. Most studies were at high RoB. Given the paucity of research in this area, it is important to encourage good quality research and highlight the areas of concern. There are five relatively simple ways to potentially improve the risk of bias of rare disease systematic reviews: 1) report a full search strategy; 2) do not apply language restrictions; 3) include conference abstracts; 4) use an appropriate RoB tool; and 5) avoid inappropriate pooling. It is not sufficient to state a study is PRISMA compliant; compliance must be demonstrated.
Patient or healthcare consumer involvement:
None