Article type
Year
Abstract
Background: Study selection is time and resource intensive. Using machine learning to identify studies that do not meet the inclusion criteria can decrease time use. The Cochrane randomized controlled trial (RCT) classifier has been trained to recognize and classify RCTs into two categories: likely or unlikely to be an RCT. The quick identification of references—for example, RCTs, which are unlikely to meet the inclusion criteria of a qualitative evidence synthesis (QES)—can speed up the study selection process.
Objectives: The first objective of this study is to evaluate whether we can use the Cochrane RCT classifier to identify nonqualitative studies that can be excluded or deprioritized during screening for a QES. The second objective is to evaluate whether there is a cutoff point in the likely-to-be-an-RCT category beyond which no qualitative studies are lost.
Methods: Using a preidentified number of QES, we will perform the following:
- extract the included studies;
- run them through the Cochrane RCT Classifier in EPPI Reviewer
- map studies that are predicted “likely to be an RCT”
- create a combined map of the predictions for all included studies across the included QES; and
- explore the studies that are classified as having a high likelihood of being an RCT to determine how they are different from the other included studies that were not given this classification.
Results: This retrospective evaluation is ongoing (completion June 2023). The results will be the map over qualitative studies that were classified as likely to be an RCT as well as the overview of how these studies are different from the studies that did not receive this classification.
Conclusions: Based on the mapping of the studies, we will conclude whether there is a threshold that reviewers can apply to automatically exclude or deprioritize studies based on their high likelihood of being an RCT. This would enable reviewers to screen faster and prioritize certain search hits for screening above others.
Patient, public and/or healthcare consumer involvement: Speeding up the review process allows evidence to become available faster to consumers. Qualitative research brings consumer perspectives forward.
Objectives: The first objective of this study is to evaluate whether we can use the Cochrane RCT classifier to identify nonqualitative studies that can be excluded or deprioritized during screening for a QES. The second objective is to evaluate whether there is a cutoff point in the likely-to-be-an-RCT category beyond which no qualitative studies are lost.
Methods: Using a preidentified number of QES, we will perform the following:
- extract the included studies;
- run them through the Cochrane RCT Classifier in EPPI Reviewer
- map studies that are predicted “likely to be an RCT”
- create a combined map of the predictions for all included studies across the included QES; and
- explore the studies that are classified as having a high likelihood of being an RCT to determine how they are different from the other included studies that were not given this classification.
Results: This retrospective evaluation is ongoing (completion June 2023). The results will be the map over qualitative studies that were classified as likely to be an RCT as well as the overview of how these studies are different from the studies that did not receive this classification.
Conclusions: Based on the mapping of the studies, we will conclude whether there is a threshold that reviewers can apply to automatically exclude or deprioritize studies based on their high likelihood of being an RCT. This would enable reviewers to screen faster and prioritize certain search hits for screening above others.
Patient, public and/or healthcare consumer involvement: Speeding up the review process allows evidence to become available faster to consumers. Qualitative research brings consumer perspectives forward.