Article type
Year
Abstract
Background:
The Cochrane Register of Studies (CRS) facilitates the management of specialised registers. It includes a set of studification routines that attempt to link related references to existing studies. Each reference is identified as potentially belonging to an existing study, to many existing studies, or no study (in which case the CRS suggests that a new study should be created).
Objectives:
This study sought to test the studification function in the CRS. It sought to answer the question: How accurate is the CRS at suggesting correct candidate records? Measures of accuracy were sensitivity - how well does the CRS correctly match references to the correct existing studies, and specificity - how well does the CRS identify when a new study needs to be created for a reference record?
Methods:
One-hundred and fifty (150) consecutive references were run through the CRS studification tab. In order to provide a comparison, all 150 records were also ‘manually’ studified. This latter form of studification acted as a gold standard and involved obtaining the full text to determine whether the study already existed in the segment or not.
Results:
On the default settings, the CRS achieved 50% sensitivity and 87% specificity. It proved 100% sensitive when a trial registration number was reported.
Conclusions:
Accurate auto-studification offers huge time-saving potential on what has largely been a fairly manual, time-intensive process. At present the CRS, on the default settings, offers very good specificity – in other words it is likely to suggest correctly that a new study should be created for records that are not related to existing studies. However, on the sample data set it only identified half the records that actually belonged to already existing studies, suggesting that this is an important area for further development.
The Cochrane Register of Studies (CRS) facilitates the management of specialised registers. It includes a set of studification routines that attempt to link related references to existing studies. Each reference is identified as potentially belonging to an existing study, to many existing studies, or no study (in which case the CRS suggests that a new study should be created).
Objectives:
This study sought to test the studification function in the CRS. It sought to answer the question: How accurate is the CRS at suggesting correct candidate records? Measures of accuracy were sensitivity - how well does the CRS correctly match references to the correct existing studies, and specificity - how well does the CRS identify when a new study needs to be created for a reference record?
Methods:
One-hundred and fifty (150) consecutive references were run through the CRS studification tab. In order to provide a comparison, all 150 records were also ‘manually’ studified. This latter form of studification acted as a gold standard and involved obtaining the full text to determine whether the study already existed in the segment or not.
Results:
On the default settings, the CRS achieved 50% sensitivity and 87% specificity. It proved 100% sensitive when a trial registration number was reported.
Conclusions:
Accurate auto-studification offers huge time-saving potential on what has largely been a fairly manual, time-intensive process. At present the CRS, on the default settings, offers very good specificity – in other words it is likely to suggest correctly that a new study should be created for records that are not related to existing studies. However, on the sample data set it only identified half the records that actually belonged to already existing studies, suggesting that this is an important area for further development.