Article type
Year
Abstract
Background: Health in my Language (HimL) is an EU-funded, three-year project. It aims to address the need for reliable and affordable translation of public health content into different languages via fully automatic machine translation (MT) systems, initially testing with translation from English into Czech, Polish, Romanian and German. Recent advances in MT are used, including in domain adaptation, translation into morphologically rich languages, terminology management, and semantically enhanced MT. Cycles of incorporating improvements into the MT systems are being iterated annually, with careful evaluation and user acceptance testing. Health information produced by Cochrane and NHS24 (Scotland's national tele-health and tele-care organisation) serves as the test case, and will be translated in each cycle and also published on their websites.
Objectives: To evaluate the quality and to test the usability of the obtained machine translations; and to measure the effect on post-editing and web access.
Methods: Different automatic evaluation metrics are applied to assess quality. The planned human evaluation tasks are:
- annotation of semantic components to assess accuracy;
- ranking of MTs against each other and human translation;
- text gap-filling to assess comprehension;
- online survey to assess user acceptance;
- post-editing of MTs to measure speed compared to post-editing of baseline MTs and fully manual translation.
Web usage statistics will be collected to assess the effect on website access of the published MTs.
Results and conclusions: The first version of the MT system was deployed in September 2015, and human semantic annotation as well as automatic metrics applied. Results varied between different text types of Cochrane and NHS24. The annotation provided some guidance for the next iteration of system development. The second system will be deployed in September 2016. The 2015 evaluation results will be presented at the Colloquium, as well as preliminary results that are available from the 2016 evaluation. The focus will be on Cochrane content.
Objectives: To evaluate the quality and to test the usability of the obtained machine translations; and to measure the effect on post-editing and web access.
Methods: Different automatic evaluation metrics are applied to assess quality. The planned human evaluation tasks are:
- annotation of semantic components to assess accuracy;
- ranking of MTs against each other and human translation;
- text gap-filling to assess comprehension;
- online survey to assess user acceptance;
- post-editing of MTs to measure speed compared to post-editing of baseline MTs and fully manual translation.
Web usage statistics will be collected to assess the effect on website access of the published MTs.
Results and conclusions: The first version of the MT system was deployed in September 2015, and human semantic annotation as well as automatic metrics applied. Results varied between different text types of Cochrane and NHS24. The annotation provided some guidance for the next iteration of system development. The second system will be deployed in September 2016. The 2015 evaluation results will be presented at the Colloquium, as well as preliminary results that are available from the 2016 evaluation. The focus will be on Cochrane content.