Article type
Year
Abstract
Background:
We are on the cusp of an important change in the way systematic reviews are produced. Artificial Intelligence (AI) and Natural Language Processing (NLP) will soon be assisting with our screening and data extraction processes. The open question remains how and where these technologies will be applied in order to yield the most value to those that produce and use systematic reviews (SRs) without jeopardizing quality, transparency and reproducibility.
Objectives:
The learning objectives of this workshop are to:
- develop a working knowledge of AI and NLP in the context of reference screening and data extraction;
- understand the current thinking on where AI fits in the systematic review process;
- understand the current limitations of AI and NLP;
- develop knowledge of the various NLP-based tools available to assist with reference screening and data extraction;
- provide an opportunity to practice using AI in the review process.
Description:
The workshop will start with a brief didactic session designed to give participants a solid baseline for the theory behind AI, NLP and classifiers in the SR context. The session will provide an overview of available research on the subject, focusing on the practical implications of the technology. The didactic session will conclude with a review of available technologies and tools that reviewers can experiment with as they explore the potential of AI for use in their own reviews.
The interactive portion of the workshop will allow participants to experiment with different applications of AI technology in the review process. The DistillerSR platform will be used as the training tool for this session and attendees will be provided with accounts and sample projects on which to work.
Attendees will do the following during the ‘hands-on’ portion of the workshop:
- experiment with training and AI to perform screening;
- use an AI to rank references in order of likeliness for inclusion;
- use an AI to assist with screening;
- use an AI review screened references for false exclusions.
Importantly, the session will examine best practices for training NLP classifiers and for developing AI-friendly protocols. Various methods for limiting the impact of potential AI errors will also be discussed.
We are on the cusp of an important change in the way systematic reviews are produced. Artificial Intelligence (AI) and Natural Language Processing (NLP) will soon be assisting with our screening and data extraction processes. The open question remains how and where these technologies will be applied in order to yield the most value to those that produce and use systematic reviews (SRs) without jeopardizing quality, transparency and reproducibility.
Objectives:
The learning objectives of this workshop are to:
- develop a working knowledge of AI and NLP in the context of reference screening and data extraction;
- understand the current thinking on where AI fits in the systematic review process;
- understand the current limitations of AI and NLP;
- develop knowledge of the various NLP-based tools available to assist with reference screening and data extraction;
- provide an opportunity to practice using AI in the review process.
Description:
The workshop will start with a brief didactic session designed to give participants a solid baseline for the theory behind AI, NLP and classifiers in the SR context. The session will provide an overview of available research on the subject, focusing on the practical implications of the technology. The didactic session will conclude with a review of available technologies and tools that reviewers can experiment with as they explore the potential of AI for use in their own reviews.
The interactive portion of the workshop will allow participants to experiment with different applications of AI technology in the review process. The DistillerSR platform will be used as the training tool for this session and attendees will be provided with accounts and sample projects on which to work.
Attendees will do the following during the ‘hands-on’ portion of the workshop:
- experiment with training and AI to perform screening;
- use an AI to rank references in order of likeliness for inclusion;
- use an AI to assist with screening;
- use an AI review screened references for false exclusions.
Importantly, the session will examine best practices for training NLP classifiers and for developing AI-friendly protocols. Various methods for limiting the impact of potential AI errors will also be discussed.