Article type
Year
Abstract
Background: Conducting systematic reviews (SRs) is frequently a resource- and time-intensive process. Many SRs are outdated even before they are published. As new research continues to become available at a fast pace, SRs constantly need to be updated in order to stay relevant. We have recently demonstrated that machine learning methods like active learning (AL) can be extremely useful in reducing the screening burden for a new review; here, we demonstrate that for the purpose of updating an existing review, the savings can potentially be even greater.
Objectives: To test if the original screening results for a systematic review can be used as a ‘seed’ to bootstrap AL when conducting a review update.
Methods: We evaluated our AL method on three SRs that expert reviewers had previously screened. To simulate a review update scenario, each dataset was divided into studies occurring before and after a chosen publication date, with studies occurring after the cut-off date used to simulate a review update. We compared standard AL on the update dataset with AL supplemented by using the prior studies as a training seed to initialize the learning model.
Results: When AL was used for the update, AL models seeded with screening results from the original review resulted in an additional 33% reduction in screening burden above the savings achieved when using AL without a seed. Furthermore, in all three cases, the recall obtained was 100%.
Conclusions: Although the cost of updating an SR can be substantial, these results demonstrate that AL models can reduce the time and cost associated with that task without reducing the accuracy. In addition, having the screening results from the original review can be very advantageous when they are used as an initial training seed for active learning methods.
Objectives: To test if the original screening results for a systematic review can be used as a ‘seed’ to bootstrap AL when conducting a review update.
Methods: We evaluated our AL method on three SRs that expert reviewers had previously screened. To simulate a review update scenario, each dataset was divided into studies occurring before and after a chosen publication date, with studies occurring after the cut-off date used to simulate a review update. We compared standard AL on the update dataset with AL supplemented by using the prior studies as a training seed to initialize the learning model.
Results: When AL was used for the update, AL models seeded with screening results from the original review resulted in an additional 33% reduction in screening burden above the savings achieved when using AL without a seed. Furthermore, in all three cases, the recall obtained was 100%.
Conclusions: Although the cost of updating an SR can be substantial, these results demonstrate that AL models can reduce the time and cost associated with that task without reducing the accuracy. In addition, having the screening results from the original review can be very advantageous when they are used as an initial training seed for active learning methods.