Article type
Year
Abstract
Background. Because they are time- and resource-intensive to produce, decision-makers often rely on sytematic reviews (SRs) that are out of date. Living SRs (LSRs) offer a potential solution by locating and incorporating new evidence in real time; however, the expedited methods needed to maintain LSRs are not well established.
Objective. To explore the value of 3 complementary search approaches in the context of LSRs in terms of search performance, screening workload, and feasibility compared to the reference standard.
Methods. A research librarian developed 3 complementary search approaches for a SR on 5 pharmacologic treatments for bronchiolitis: the automated full search, Pubmed Similar Articles, and Scopus Citing Articles. Beginning October 2018, the searches were automated to run monthly (except Pubmed Similar Articles, which was performed manually). Pairs of reviewers (one per search approach) independently screened records retrieved via each search and commented on feasibility monthly. After one year, we conducted a full update search in four online databases, selected conference proceedings and clinical trials registers (reference standard). For each complementary approach, we calculated search performance (proportion missed, number needed to read [NNR]) and reviewer workload (number of records screened, time required) compared to the reference standard. We summarized comments about feasibility. We will investigate the impact of the newly located trials on the effect estimate and certainty of evidence for the 2 primary outcomes.
Results. Via the reference standard, reviewers screened 505 titles/abstracts and 24 full texts and identified 4 new trials (NNR 126.6; 12.4 hours). Of the complementary approaches, only the automated full search located all 4 trials; these were located 5 to 12 months sooner than via the reference standard. The automated full search was the most resource-intensive approach (NNR 204.1; 17.0 hours). Compared to the reference standard, reviewers screened more records (811 titles/abstracts and 21 full texts), due to duplicates. The Pubmed Similar Articles and Scopus Citing Articles approaches located far fewer candidate records (244 and 451, respectively), thereby requiring less screening time (5.9 and 8.7 hours, respectively); however, each approach located only 1 of the 4 new trials (75% missed). Reviewers found it feasible and convenient to conduct monthly screening for searches of this yield (median 15 to 65 records/month).
Conclusions. The automated full search located relevant trials sooner, but required more screening time than the reference standard. Although the monthly screening time for the Pubmed Similar Articles and Scopus Citing Articles was far less, most relevant records were missed. Exploration of the impact of locating the 4 trials on the results and conclusions of the SR will provide additional insight into the value of each approach.
Patient or consumer involvement. Consumers were not directly involved, but will benefit from LSR methods that ensure the availability of up-to-date evidence.
Objective. To explore the value of 3 complementary search approaches in the context of LSRs in terms of search performance, screening workload, and feasibility compared to the reference standard.
Methods. A research librarian developed 3 complementary search approaches for a SR on 5 pharmacologic treatments for bronchiolitis: the automated full search, Pubmed Similar Articles, and Scopus Citing Articles. Beginning October 2018, the searches were automated to run monthly (except Pubmed Similar Articles, which was performed manually). Pairs of reviewers (one per search approach) independently screened records retrieved via each search and commented on feasibility monthly. After one year, we conducted a full update search in four online databases, selected conference proceedings and clinical trials registers (reference standard). For each complementary approach, we calculated search performance (proportion missed, number needed to read [NNR]) and reviewer workload (number of records screened, time required) compared to the reference standard. We summarized comments about feasibility. We will investigate the impact of the newly located trials on the effect estimate and certainty of evidence for the 2 primary outcomes.
Results. Via the reference standard, reviewers screened 505 titles/abstracts and 24 full texts and identified 4 new trials (NNR 126.6; 12.4 hours). Of the complementary approaches, only the automated full search located all 4 trials; these were located 5 to 12 months sooner than via the reference standard. The automated full search was the most resource-intensive approach (NNR 204.1; 17.0 hours). Compared to the reference standard, reviewers screened more records (811 titles/abstracts and 21 full texts), due to duplicates. The Pubmed Similar Articles and Scopus Citing Articles approaches located far fewer candidate records (244 and 451, respectively), thereby requiring less screening time (5.9 and 8.7 hours, respectively); however, each approach located only 1 of the 4 new trials (75% missed). Reviewers found it feasible and convenient to conduct monthly screening for searches of this yield (median 15 to 65 records/month).
Conclusions. The automated full search located relevant trials sooner, but required more screening time than the reference standard. Although the monthly screening time for the Pubmed Similar Articles and Scopus Citing Articles was far less, most relevant records were missed. Exploration of the impact of locating the 4 trials on the results and conclusions of the SR will provide additional insight into the value of each approach.
Patient or consumer involvement. Consumers were not directly involved, but will benefit from LSR methods that ensure the availability of up-to-date evidence.