Article type
Abstract
Background
The Cochrane Central Register of Controlled Trials (CENTRAL) is compiled from a number of sources. Since 2017, we have increased the number of sources feeding CENTRAL and improved the efficiency of our processes through the use of APIs, machine learning and crowdsourcing, known as the Cochrane Evidence Pipeline. Our objectives were: (1) Assess the effectiveness of Cochrane’s Evidence Pipeline to identify references to published reports eligible for inclusion in Cochrane reviews of randomised controlled trials (RCTs); (2) Identify opportunities to improve performance of Cochrane's Evidence Pipeline.
Methods
This work is based on two analyses covering two time periods. For the first analysis, we identified all references to RCTs with a publication or registration date between 1st January 2017 and 31st December 2018 that had been included in a Cochrane intervention review. We then viewed the audit trail for each reference to determine if it had been identified by the Evidence Pipeline and subsequently added to CENTRAL. The second analysis, currently underway, will follow the same methods but cover the timeframe from 2019 to 2024.
Results
From the first analysis, we identified 650 references to included studies with a publication year of 2017 or 2018. Of those, 634 (97.5%) had been captured by the Evidence Pipeline. Sixteen references had been missed: six had PubMed-not-MEDLINE status, four were missed by the centralised Embase search, three had been misclassified by Cochrane Crowd, one was from a journal not indexed in MEDLINE or Embase, one had only been added to Embase in 2019, and one reference had been rejected by the RCT machine learning classifier. Of the missed references, eight were the main or only publication to the trial in the review in which it had been included. Results from the second analysis will be presented at the Global Evidence Summit.
Conclusions
Results from the first analysis show that Cochrane’s Evidence Pipeline is highly sensitive. The results also helped us to understand why some RCTs had been missed. The Cochrane Evidence Pipeline is playing a critical role in helping to populate CENTRAL and is moving towards making CENTRAL a comprehensive repository of RCTs.
The Cochrane Central Register of Controlled Trials (CENTRAL) is compiled from a number of sources. Since 2017, we have increased the number of sources feeding CENTRAL and improved the efficiency of our processes through the use of APIs, machine learning and crowdsourcing, known as the Cochrane Evidence Pipeline. Our objectives were: (1) Assess the effectiveness of Cochrane’s Evidence Pipeline to identify references to published reports eligible for inclusion in Cochrane reviews of randomised controlled trials (RCTs); (2) Identify opportunities to improve performance of Cochrane's Evidence Pipeline.
Methods
This work is based on two analyses covering two time periods. For the first analysis, we identified all references to RCTs with a publication or registration date between 1st January 2017 and 31st December 2018 that had been included in a Cochrane intervention review. We then viewed the audit trail for each reference to determine if it had been identified by the Evidence Pipeline and subsequently added to CENTRAL. The second analysis, currently underway, will follow the same methods but cover the timeframe from 2019 to 2024.
Results
From the first analysis, we identified 650 references to included studies with a publication year of 2017 or 2018. Of those, 634 (97.5%) had been captured by the Evidence Pipeline. Sixteen references had been missed: six had PubMed-not-MEDLINE status, four were missed by the centralised Embase search, three had been misclassified by Cochrane Crowd, one was from a journal not indexed in MEDLINE or Embase, one had only been added to Embase in 2019, and one reference had been rejected by the RCT machine learning classifier. Of the missed references, eight were the main or only publication to the trial in the review in which it had been included. Results from the second analysis will be presented at the Global Evidence Summit.
Conclusions
Results from the first analysis show that Cochrane’s Evidence Pipeline is highly sensitive. The results also helped us to understand why some RCTs had been missed. The Cochrane Evidence Pipeline is playing a critical role in helping to populate CENTRAL and is moving towards making CENTRAL a comprehensive repository of RCTs.