Improving search efficiency for economic evaluations in major databases using semantic technology

Article type
Authors
May-Glanville J1, Lefebvre C2, Porter B3, Negosanti P4
1York Health Economics Consortium, York, UK
2UK Cochrane Centre, Oxford, UK
3Expert System, Lymington, UK
4Expert System, Modena, Italy
Abstract
Background: Increasingly systematic reviews seek evidence from study designs which are hard to identify efficiently: economic evaluations, adverse effects reports, observational studies, quality of life studies and diagnostic test accuracy studies. Semantic technology software understands automatically the meaning of text written in natural language and may offer approaches to improve the retrieval of difficult to identify study designs. One such issue for systematic reviewers is identifying economic evaluations. Searching for economic evaluations is problematic because they are difficult to distinguish efficiently from other economic studies. Objectives: This research explores whether semantic technology post-processing software can help to improve search efficiency for difficult to identify study designs such as economic evaluations. Methods: We identified a gold-standard set of economic evaluation records from the NHS EED database: cost-benefit studies, cost-effectiveness studies and cost-utility studies. We identified a comparison set of records of other forms of economic study from MEDLINE. Using these two sets of records we trained semantic technology software to discriminate economic evaluations from other economic studies. We tested the performance of the semantic technology software in identifying economic evaluations accurately and then adapted the software to obtain further improvements. Results: Initial testing produced 90% sensitivity (recall) and 85% precision. The final results will be presented at the conference. Conclusions: Initial testing shows promise for improving the precision of searching for economic evaluations among records identified in a major bibliographic database. The potential for improved efficiency may be even greater in databases such as EMBASE where extensive indexing seems to impede efficient retrieval. Semantic technology, for post-processing of search results achieved from sensitive searches, may offer a solution to the current challenges of identifying ‘hard to focus’ study designs such as economic evaluations, adverse effects reports, observational studies, quality of life studies and diagnostic test accuracy studies.