Article type
Year
Abstract
Background:
Systematic reviews search several databases. Search results then need to be de-duplicated. Generally, reviewers de-duplicate using bibliographic software. Using software with the most efficient automatic de-duplication algorithm would save time.
Objectives:
To identify the most efficient de-duplication options from six commonly used and/or free of charge software packages. Our criteria for efficiency was 100% specificity and the highest possible sensitivity. 100% specificity is critical to avoid potentially relevant studies being incorrectly de-duplicated - we can compensate for poor sensitivity by manual de-duplication.
Methods:
We manually de-duplicated the search results of two literature searches (1,578 mammography records and 2,239 atopic dermatitis records) within Endnote X9 to produce two sets of known duplicates against which to compare the performance of the selected software. We ran these results through the automatic de-duplication options available in Citavi, Endnote, EPPI-Reviewer 5 (beta), Mendeley, the Systematic Review Assistant-DeDuplication Module (SRA-DM) and Zotero. We calculated the specificity and sensitivity of the software.
Results:
The results are in Tables 1 and 2. According to our criteria, the most efficient software is Endnote (100% specificity). Mendeley and Citavi exceeded Endnote’s sensitivity in both result sets, but had lower specificity in the atopic dermatitis set (99.8% and 99.3% respectively). SRA-DM was less sensitive and less specific than Endnote, and EPPI-Reviewer did not achieve 100% specificity in either record set.
Conclusions:
Endnote was the most efficient software package according to our specifications with 100% specificity. Endnote’s slightly lower sensitivity compared to some other software can be redressed with manual de-duplication, whereas records incorrectly de-duplicated by other software may not be easily recovered. Zotero seemed the least safe option, de-duplicating all records from the same abstract book.
Efficiency is not the only benefit of de-duplication software. When sensitivity is lower than 100% manual de-duplication is also required. However, the relative advantages of conducting manual de-duplication in the different software packages was beyond our scope. Ancillary approaches to manual de-duplication such as published algorithms1 were not assessed for this project. Further, we did not assess software features that mitigate low specificity (e.g. Citavi’s manual review of identified duplicates) since they introduced a second step to de-deduplication. Finally, each software has default de-duplication algorithms, and we did not assess the impact of choosing a different default, where available.
Due to resource constraints we only used two test sets, which means generalisability may be impacted. A larger number of result sets would help with generalisability.
Patient or healthcare consumer involvement:
This research was methods focused and no patients were involved. We are the consumers of the software products.
Systematic reviews search several databases. Search results then need to be de-duplicated. Generally, reviewers de-duplicate using bibliographic software. Using software with the most efficient automatic de-duplication algorithm would save time.
Objectives:
To identify the most efficient de-duplication options from six commonly used and/or free of charge software packages. Our criteria for efficiency was 100% specificity and the highest possible sensitivity. 100% specificity is critical to avoid potentially relevant studies being incorrectly de-duplicated - we can compensate for poor sensitivity by manual de-duplication.
Methods:
We manually de-duplicated the search results of two literature searches (1,578 mammography records and 2,239 atopic dermatitis records) within Endnote X9 to produce two sets of known duplicates against which to compare the performance of the selected software. We ran these results through the automatic de-duplication options available in Citavi, Endnote, EPPI-Reviewer 5 (beta), Mendeley, the Systematic Review Assistant-DeDuplication Module (SRA-DM) and Zotero. We calculated the specificity and sensitivity of the software.
Results:
The results are in Tables 1 and 2. According to our criteria, the most efficient software is Endnote (100% specificity). Mendeley and Citavi exceeded Endnote’s sensitivity in both result sets, but had lower specificity in the atopic dermatitis set (99.8% and 99.3% respectively). SRA-DM was less sensitive and less specific than Endnote, and EPPI-Reviewer did not achieve 100% specificity in either record set.
Conclusions:
Endnote was the most efficient software package according to our specifications with 100% specificity. Endnote’s slightly lower sensitivity compared to some other software can be redressed with manual de-duplication, whereas records incorrectly de-duplicated by other software may not be easily recovered. Zotero seemed the least safe option, de-duplicating all records from the same abstract book.
Efficiency is not the only benefit of de-duplication software. When sensitivity is lower than 100% manual de-duplication is also required. However, the relative advantages of conducting manual de-duplication in the different software packages was beyond our scope. Ancillary approaches to manual de-duplication such as published algorithms1 were not assessed for this project. Further, we did not assess software features that mitigate low specificity (e.g. Citavi’s manual review of identified duplicates) since they introduced a second step to de-deduplication. Finally, each software has default de-duplication algorithms, and we did not assess the impact of choosing a different default, where available.
Due to resource constraints we only used two test sets, which means generalisability may be impacted. A larger number of result sets would help with generalisability.
Patient or healthcare consumer involvement:
This research was methods focused and no patients were involved. We are the consumers of the software products.