Objectives: This workshop will examine the use of RobotReviewer to make the data-extraction stage of systematic reviews more efficient via semi-automation. We will provide hands-on experience with the tool, and lead discussion of the methodological implications of using semi-automated approaches to support systematic reviews.

Description: RobotReviewer is a system that uses machine learning to automate data extraction and synthesis from clinical trial reports. Currently, RR is capable of automatically extracting information on study design, characteristics of the population, interventions, and outcomes, and assessing risks of bias.

In this workshop, we will introduce RobotReviewer and demonstrate its use in practice for producing both semi-automated (wherein a human checks the generated output) and fully automatic evidence summaries. We will describe the underlying methods for automation, and we will use examples to explore the impact this can have on real-world use.

Finally, we will discuss barriers and solutions to using automation in practice. We will additionally present a pilot study of RobotReviewer, which will enable participating review authors to evaluate the tool’s usability, and receive real-time feedback on accuracy and time taken on their particular project. Participants should bring a laptop. Participants actively working on a systematic review or guideline are encouraged to bring along some example clinical trial PDFs to try out with RobotReviewer.