Article type
Year
Abstract
Background
To facilitate quick appraisal of the usefulness of systematic reviews (SRs), the Swedish Agency for Health Technology Assessment (HTA) and Assessment of Social Services (SBU) has developed SNABBSTAR. This tool, based on A MeaSurement Tool to Assess systematic Reviews (AMSTAR), implies a stepwise assessment of the methodological/reporting quality. The six steps concern the following: 1. definition of population, intervention, control, and outcomes (PICO), literature search; 2. inclusion/exclusion according to PICO, listing of included studies; 3. risk of bias assessments; 4. evidence synthesis/meta-analyses; and 5. certainty of evidence considerations; 6. documentation of excluded studies, conflicts of interest, and SR protocol. Assessment is stopped if an SR does not meet the quality criteria of a step. SRs not fulfilling the first step are considered not useful, and those reaching the fifth step and beyond can be used in HTAs with limited or no complementary work.
Objectives
To apply SNABBSTAR to evaluate the usefulness of SRs systematically identified in an HTA for evidence-based decision-making.
Methods
We included all SRs published in 2018-2020 identified in the literature search in four databases, and fulfilling the PICO, of an HTA (Sahlgrenska University Hospital, Gothenburg, Sweden, 2021:123) investigating efficacy and safety outcomes for clopidogrel versus ticagrelor in patients subjected to dual antiplatelet therapy in acute coronary syndrome. Four assessors independently assessed all SRs using SNABBSTAR, and diverging assessments were resolved in consensus discussions.
Results
A total of 13 SRs were assessed (China, n=7; Italy, n=2; USA, n=2; Poland, n=1; Kazakstan, n=1), 10 of which were described to be reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The SRs included 0‒16 randomised controlled trials (RCTs) and 0‒8 non-RCTs. In all, 12 SRs were assessed to fail already at step 1 in SNABBSTAR; the literature search could not be reproduced with the information provided. One SR passed step 4 but not step 5 because the certainty of evidence was not reflected in the conclusions.
Conclusion
This analysis suggests that SRs often fail to meet fundamental quality criteria required to be useful in evidence-based decision-making; the reporting of the literature search process deserves further attention in SR publications. A stepwise appraisal, starting with the definition of the PICO/literature search, may facilitate quick assessments of the usefulness of SRs.
To facilitate quick appraisal of the usefulness of systematic reviews (SRs), the Swedish Agency for Health Technology Assessment (HTA) and Assessment of Social Services (SBU) has developed SNABBSTAR. This tool, based on A MeaSurement Tool to Assess systematic Reviews (AMSTAR), implies a stepwise assessment of the methodological/reporting quality. The six steps concern the following: 1. definition of population, intervention, control, and outcomes (PICO), literature search; 2. inclusion/exclusion according to PICO, listing of included studies; 3. risk of bias assessments; 4. evidence synthesis/meta-analyses; and 5. certainty of evidence considerations; 6. documentation of excluded studies, conflicts of interest, and SR protocol. Assessment is stopped if an SR does not meet the quality criteria of a step. SRs not fulfilling the first step are considered not useful, and those reaching the fifth step and beyond can be used in HTAs with limited or no complementary work.
Objectives
To apply SNABBSTAR to evaluate the usefulness of SRs systematically identified in an HTA for evidence-based decision-making.
Methods
We included all SRs published in 2018-2020 identified in the literature search in four databases, and fulfilling the PICO, of an HTA (Sahlgrenska University Hospital, Gothenburg, Sweden, 2021:123) investigating efficacy and safety outcomes for clopidogrel versus ticagrelor in patients subjected to dual antiplatelet therapy in acute coronary syndrome. Four assessors independently assessed all SRs using SNABBSTAR, and diverging assessments were resolved in consensus discussions.
Results
A total of 13 SRs were assessed (China, n=7; Italy, n=2; USA, n=2; Poland, n=1; Kazakstan, n=1), 10 of which were described to be reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The SRs included 0‒16 randomised controlled trials (RCTs) and 0‒8 non-RCTs. In all, 12 SRs were assessed to fail already at step 1 in SNABBSTAR; the literature search could not be reproduced with the information provided. One SR passed step 4 but not step 5 because the certainty of evidence was not reflected in the conclusions.
Conclusion
This analysis suggests that SRs often fail to meet fundamental quality criteria required to be useful in evidence-based decision-making; the reporting of the literature search process deserves further attention in SR publications. A stepwise appraisal, starting with the definition of the PICO/literature search, may facilitate quick assessments of the usefulness of SRs.