Article type
Abstract
Background: As part of the Cochrane Linked data project, we are annotating all Cochrane systematic reviews. Phase I, which ends in March 2017, involves annotating child-health reviews. The reviews are being annotated at two levels: review level (i.e. the question) and included study level (generating a PICO for each study). This work is part of a project funded by the Bill and Melinda Gates Foundation to generate a PICO-annotated evidence base of all Pregnancy and Neonatal Cochrane Reviews.
Similarly, Cochrane Crowd (http://crowd.cochrane.org) has now rolled out a beta PICO task. This crowd task is aimed at surfacing the core PICO elements from reports of randomised controlled trials in the areas of pregnancy and child health. The outputs of each will feed the PICO concept browser and PICO Finder: two user interfaces that facilitate discovery of Cochrane Reviews and their included trials according to PICO.
Objectives: 1. To annotate all reviews relevant to child health using the Linked data annotator tool and ontology developed by the Cochrane Linked data team.
2. To create a Crowd PICO task that would enable high-quality annotation of titles and abstracts by a Crowd.
Methods: For the Crowd annotation, a search was run to identify potentially relevant reports of trials that had not yet been included in Cochrane reviews. We fed these citations into the beta Crowd PICO task. We developed a training module to help guide Crowd contributors through an interactive example. We then made the task available to all those who had screened a certain number of records for the RCT task.
For the annotation of reviews, we focused our efforts on those reviews from the Neonatal and Pregnancy review groups. We recruited a small team of annotators and worked closely with Cochrane information specialists to ensure quality and consistent annotations.
Results: We will present results on the number of annotations performed by the Crowd and Expert annotators and the quality of those annotations across the four PICO elements.
Similarly, Cochrane Crowd (http://crowd.cochrane.org) has now rolled out a beta PICO task. This crowd task is aimed at surfacing the core PICO elements from reports of randomised controlled trials in the areas of pregnancy and child health. The outputs of each will feed the PICO concept browser and PICO Finder: two user interfaces that facilitate discovery of Cochrane Reviews and their included trials according to PICO.
Objectives: 1. To annotate all reviews relevant to child health using the Linked data annotator tool and ontology developed by the Cochrane Linked data team.
2. To create a Crowd PICO task that would enable high-quality annotation of titles and abstracts by a Crowd.
Methods: For the Crowd annotation, a search was run to identify potentially relevant reports of trials that had not yet been included in Cochrane reviews. We fed these citations into the beta Crowd PICO task. We developed a training module to help guide Crowd contributors through an interactive example. We then made the task available to all those who had screened a certain number of records for the RCT task.
For the annotation of reviews, we focused our efforts on those reviews from the Neonatal and Pregnancy review groups. We recruited a small team of annotators and worked closely with Cochrane information specialists to ensure quality and consistent annotations.
Results: We will present results on the number of annotations performed by the Crowd and Expert annotators and the quality of those annotations across the four PICO elements.