The Polyglot Search Translator (PST): evaluation of a tool for improving searching in systematic reviews: a randomised cross-over trial

Article type
Authors
Clark J1, Carter M1, Honeyman D2, Cleo G1, Auld Y3, Booth D4, Condron P5, Dalais C6, Dern S2, Linthwaite B3, May N7, Munn J8, Ramsay L9, Rickett K6, Rutter C10, Sanders S1, Sondergeld P10, Wallin M11, Beller E1
1Centre for Research in Evidence Based Practice, Bond University
2Bond University Library, Bond University
3Gold Coast Health Library Service, Gold Coast University Hospital
4University Library, University of Newcastle
5University Library, University of Melbourne
6University Library, University of Queensland
7Sturt Library, Flinders University
8Centre for Teaching and Learning, Southern Cross University
9Library Academic and Research Services, Australian Catholic University
10University Library, Queensland University of Technology
11University Library, Southern Cross University
Abstract
Background:
To improve the speed and quality of literature searching for systematic reviews (SRs) we developed an online tool, the Polyglot Search Translator (PST). This tool helps to automatically translate searches across multiple databases by modifying the database-specific syntax, removing the need to do this manually.

Objectives:
To determine whether the PST improves the speed and quality of database searching for SRs.

Methods:
From a pool of 20 randomly selected database search strategies from reviews of drug and non-drug interventions, diagnostic tests, prevalence, prognosis and health technology assessments, each of 20 participants were randomly assigned 10 searches and randomly allocated to translate the search manually or with the PST. The participants were asked to translate each search into three databases from the following: PubMed or Ovid MEDLINE, Embase.com or Ovid Embase, the Cochrane Library, CINAHL, Web of Science or Scopus. We measured:

1) the time taken to translate the search strategies using manual methods or the PST (these were compared against each other);
2) the number of search results found by the participants, compared to the number found by a reference standard developed through consensus by two Cochrane information specialists;
3) errors found in the participants' searches, identified through consensus by the two Cochrane information specialists (manual errors will be compared to PST errors).

Results:
Preliminary results show that when compared with a manual method, using the PST to translate a search saved an average of 10 minutes per database; this increased to over 100 minutes per database on the most complex search. As more data become available, a more accurate representation of the time saving is expected. The number of search results found and errors in the searches are still being analysed.

Conclusions:
Initial data show that the PST saves time when searching for studies for inclusion in a SR. Analysis has not yet begun of the quality of search translations; these data will be presented.

Patient or healthcare consumer involvement:
Although not directly involving patients or health care consumers, the PST could open pathways for less experienced individuals to be involved in a SR and speed up SR production, with the benefit to consumers of having access to more up-to-date SRs.