Article type
Year
Abstract
Background:
A consortium of Metaxis, the Cochrane Dementia and Cognitive Improvement Group, and the York Health Economics Consortium responded to a call for proposals to take over management of the EMBASE project. The aim of the project was identify randomised and quasi-randomised trials from sensitive searches run in EMBASE for their inclusion in the Cochrane Central Register of Controlled Trials (CENTRAL).
Objectives:
The new project team had some immediate objectives. These included developing a new, validated, sensitive search (or filter) to be run in EMBASE (via Ovid SP). The team also needed to design and create an online citation screening tool and to formulate and implement a strategy for screening the citations through crowdsourcing.
Methods:
The search strategy was devised using various textual analysis techniques, each of which generated candidate terms tested against a large gold standard dataset. The screening tool was developed and includes: a management interface, a function to have certain words and phrases highlighted and a well-developed ‘History’ tab enabling screeners to view their decisions against other screener decisions. The recruitment of screeners was staggered to allow in-depth feedback to be sought before roll-out to the rest of the Cochrane and beyond.
Results:
To date thousands of citations have been fed into CENTRAL from the EMBASE project and tens of thousands of citations have been screened by the crowd. We will present results on the new search approach and new strategy, the performance and functionality of the screening tool, including the management tool, and the results of the crowdsourced element of the project: crowd sensitivity, crowd specificity, and motivation and incentive.
Conclusions:
This poster will illustrate the main tasks and challenges faced by the team in their first year. It is an exciting an innovative project that demonstrates the potential role of crowdsourcing in creating Cochrane content.
A consortium of Metaxis, the Cochrane Dementia and Cognitive Improvement Group, and the York Health Economics Consortium responded to a call for proposals to take over management of the EMBASE project. The aim of the project was identify randomised and quasi-randomised trials from sensitive searches run in EMBASE for their inclusion in the Cochrane Central Register of Controlled Trials (CENTRAL).
Objectives:
The new project team had some immediate objectives. These included developing a new, validated, sensitive search (or filter) to be run in EMBASE (via Ovid SP). The team also needed to design and create an online citation screening tool and to formulate and implement a strategy for screening the citations through crowdsourcing.
Methods:
The search strategy was devised using various textual analysis techniques, each of which generated candidate terms tested against a large gold standard dataset. The screening tool was developed and includes: a management interface, a function to have certain words and phrases highlighted and a well-developed ‘History’ tab enabling screeners to view their decisions against other screener decisions. The recruitment of screeners was staggered to allow in-depth feedback to be sought before roll-out to the rest of the Cochrane and beyond.
Results:
To date thousands of citations have been fed into CENTRAL from the EMBASE project and tens of thousands of citations have been screened by the crowd. We will present results on the new search approach and new strategy, the performance and functionality of the screening tool, including the management tool, and the results of the crowdsourced element of the project: crowd sensitivity, crowd specificity, and motivation and incentive.
Conclusions:
This poster will illustrate the main tasks and challenges faced by the team in their first year. It is an exciting an innovative project that demonstrates the potential role of crowdsourcing in creating Cochrane content.