Article type
Year
Abstract
Background:
Cochrane Systematic Reviews rely on the efficient identification of research evidence, specifically evidence from randomised controlled trials (RCTs). The largest single source of reports of RCTs is the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library. CENTRAL is mainly populated with records from MEDLINE, but also contains a substantial and growing number of records from Embase. The objective was to develop a new bespoke search filter to identify reports of RCTs and novel methods to assess the high volume of candidate reports resulting from the filter.
Methods: We developed, validated and refined a sensitive search filter to identify reports of RCTs in Embase. This filter was developed using textual analysis of ten gold standard sets of RCT records (totalling 10,000 records over ten years). The filter performance was tested on a second set of 10,000 RCT reports. Once implemented, records retrieved by the filter were assessed for relevance by a novel crowdsource approach. The search filter was refined after one year of operation based on an assessment of the records rejected by the crowd.
Results: The development of the search filter and the analysis of output from Embase has resulted in a tiered assessment process, where the most obvious RCT reports are fast-tracked for publication in CENTRAL, leaving more capacity to assess the relevance of less obvious candidate records. Over a 15-month period the filter has identified 198,960 records and 55,042 reports of RCTs have been added to CENTRAL (precision 28%).
Conclusions: The records identified by the filter and the crowdsource process have made many thousands of reports of RCTs that were unique to Embase, available in CENTRAL at a high level of precision. These RCTs might be otherwise inaccessible to Cochrane authors since many of them may not have access to Embase.
Cochrane Systematic Reviews rely on the efficient identification of research evidence, specifically evidence from randomised controlled trials (RCTs). The largest single source of reports of RCTs is the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library. CENTRAL is mainly populated with records from MEDLINE, but also contains a substantial and growing number of records from Embase. The objective was to develop a new bespoke search filter to identify reports of RCTs and novel methods to assess the high volume of candidate reports resulting from the filter.
Methods: We developed, validated and refined a sensitive search filter to identify reports of RCTs in Embase. This filter was developed using textual analysis of ten gold standard sets of RCT records (totalling 10,000 records over ten years). The filter performance was tested on a second set of 10,000 RCT reports. Once implemented, records retrieved by the filter were assessed for relevance by a novel crowdsource approach. The search filter was refined after one year of operation based on an assessment of the records rejected by the crowd.
Results: The development of the search filter and the analysis of output from Embase has resulted in a tiered assessment process, where the most obvious RCT reports are fast-tracked for publication in CENTRAL, leaving more capacity to assess the relevance of less obvious candidate records. Over a 15-month period the filter has identified 198,960 records and 55,042 reports of RCTs have been added to CENTRAL (precision 28%).
Conclusions: The records identified by the filter and the crowdsource process have made many thousands of reports of RCTs that were unique to Embase, available in CENTRAL at a high level of precision. These RCTs might be otherwise inaccessible to Cochrane authors since many of them may not have access to Embase.