Article type
Abstract
"Background
The volume of peer-reviewed literature is increasing exponentially in every discipline. This is leading to increased resources needed for conducting systematic reviews (SRs), including for health technology assessment (HTA) where review questions can be broad or within widely researched areas. We aimed to explore methods utilized to manage high evidence volumes in SRs, using Covid-19 vaccine research as an example.
Methods
This overview of reviews was conducted according to a prespecified protocol. In February 2023, we searched MEDLINE, Embase and Cochrane Database of Systematic Reviews for SRs focusing on efficacy or safety of Covid-19 vaccines in unselected populations. Records were reviewed by a single individual at the title and then full-text review stage. Information was extracted on whether any restrictions were placed (either a priori or a posteriori) on the search terms or characteristics of studies included in the SRs to prioritize evidence, including to any of: publication date, study design, location, sample size, vaccine type, study quality.
Results
We included 103 SRs, none of which explicitly stated using an approach to limit the analyzed evidence. However, five SRs prioritized data extractions for articles that were judged to be at sufficient study quality. 86 SRs used restrictive search terms beyond population terms, resulting in few records included via database searches and reducing screening burden, but increasing the likelihood of missing relevant data. A large proportion of SRs with high database hits excluded substantial numbers of records either before or during the title/abstract review stage, resulting in small numbers of full-texts screened, but did not report clear rationale for this.
Conclusions
Evidence screening approaches vary widely over reviews with similar topics and authors do not openly discuss issues of dealing with high volumes of evidence. Transparent reporting of prioritization approaches is needed to establish clear methods for streamlining SRs and avoid these becoming resource-intensive to the point of unsustainability resulting in significant research waste.
"
The volume of peer-reviewed literature is increasing exponentially in every discipline. This is leading to increased resources needed for conducting systematic reviews (SRs), including for health technology assessment (HTA) where review questions can be broad or within widely researched areas. We aimed to explore methods utilized to manage high evidence volumes in SRs, using Covid-19 vaccine research as an example.
Methods
This overview of reviews was conducted according to a prespecified protocol. In February 2023, we searched MEDLINE, Embase and Cochrane Database of Systematic Reviews for SRs focusing on efficacy or safety of Covid-19 vaccines in unselected populations. Records were reviewed by a single individual at the title and then full-text review stage. Information was extracted on whether any restrictions were placed (either a priori or a posteriori) on the search terms or characteristics of studies included in the SRs to prioritize evidence, including to any of: publication date, study design, location, sample size, vaccine type, study quality.
Results
We included 103 SRs, none of which explicitly stated using an approach to limit the analyzed evidence. However, five SRs prioritized data extractions for articles that were judged to be at sufficient study quality. 86 SRs used restrictive search terms beyond population terms, resulting in few records included via database searches and reducing screening burden, but increasing the likelihood of missing relevant data. A large proportion of SRs with high database hits excluded substantial numbers of records either before or during the title/abstract review stage, resulting in small numbers of full-texts screened, but did not report clear rationale for this.
Conclusions
Evidence screening approaches vary widely over reviews with similar topics and authors do not openly discuss issues of dealing with high volumes of evidence. Transparent reporting of prioritization approaches is needed to establish clear methods for streamlining SRs and avoid these becoming resource-intensive to the point of unsustainability resulting in significant research waste.
"