Article type
Year
Abstract
Background: Automation technology has been proposed or used to accelerate most steps of the systematic review process, including search, screening, data extraction, and quality evaluation. However, how many reviews are using automation now remains unclear.
Objectives: The aim of this study was to review the current situation and quality of reviews using automation.
Methods: We searched MEDLINE via PubMed, EMBASE, Cochrane Library, Web of Science from inception to 1 March 2020 for evidence synthesis studies that used the automation methods. The following search strategy was used: [Automation terms (such as RobotReviewer, machine learning, Artificial Intelligence, etc.)] AND [evidence synthesis terms (such as systematic review, meta-analysis, literature review, etc.)]. The search was no language restrictions and was limited to human subjects. We used AMSTAR to assess the quality of included studies. Finally, we conducted a descriptive analysis of the characteristic of the included reviews.
Results: A total of 61 reviews were included. More than half of the review (36; 59.0%) were published between 2019 and 2020. The topics of the included reviews varied. The most common type was systematic reviews (57; 93%), including 15 Cochrane reviews (24%). Most reviews used Covidence (41; 77%), and others used Rayyan (8; 13%), EPPI-Reviewer 4 (5; 8%) and RobotReviewer (3; 5%). The application of automation in evidence synthesis mainly included duplication removing (52; 86%), study selection (49; 81%), data-extraction (21; 35%) and quality assessment (11; 18%). The quality of those reviews was moderate to high. More than half of the reviews (40; 66%) scored more than 8 (total score 11), and the rest were between six and eight.
Conclusions: In recent years, the number of reviews using automation technology has been increasing, and the quality is relatively high. At present, it is mainly used in the retrieval, screening, extraction and evaluation stages of documents. However, the reliability and validity of automation tools should be specified.
Patient or healthcare consumer involvement: None
Objectives: The aim of this study was to review the current situation and quality of reviews using automation.
Methods: We searched MEDLINE via PubMed, EMBASE, Cochrane Library, Web of Science from inception to 1 March 2020 for evidence synthesis studies that used the automation methods. The following search strategy was used: [Automation terms (such as RobotReviewer, machine learning, Artificial Intelligence, etc.)] AND [evidence synthesis terms (such as systematic review, meta-analysis, literature review, etc.)]. The search was no language restrictions and was limited to human subjects. We used AMSTAR to assess the quality of included studies. Finally, we conducted a descriptive analysis of the characteristic of the included reviews.
Results: A total of 61 reviews were included. More than half of the review (36; 59.0%) were published between 2019 and 2020. The topics of the included reviews varied. The most common type was systematic reviews (57; 93%), including 15 Cochrane reviews (24%). Most reviews used Covidence (41; 77%), and others used Rayyan (8; 13%), EPPI-Reviewer 4 (5; 8%) and RobotReviewer (3; 5%). The application of automation in evidence synthesis mainly included duplication removing (52; 86%), study selection (49; 81%), data-extraction (21; 35%) and quality assessment (11; 18%). The quality of those reviews was moderate to high. More than half of the reviews (40; 66%) scored more than 8 (total score 11), and the rest were between six and eight.
Conclusions: In recent years, the number of reviews using automation technology has been increasing, and the quality is relatively high. At present, it is mainly used in the retrieval, screening, extraction and evaluation stages of documents. However, the reliability and validity of automation tools should be specified.
Patient or healthcare consumer involvement: None