Article type
Year
Abstract
Background: Systematic reviews are foundational for evidence-based care, but require a substantial time investment by the authorship team. There are currently no data or guidelines regarding the most effective strategy for screening citations before full-text review.
Objective: To compare title-abstract and title-only strategies for screening citations for a systematic review.
Methods: A five-member group of physicians performed a systematic review and meta-analysis on light at night and breast cancer risk using Medline and Embase (January to March 2011). Group members were assigned in a round-robin fashion to two reviewer pairs. Each reviewer assessed whether citations included 1) primary data, 2) the exposure of interest, and 3) the outcome of interest. A titles-only screening strategy was undertaken first, followed by a combined titles-abstract strategy on the same citation list. Outcome measures included inter-rater agreement and reliability and articles ultimately included for review following both screening strategies.
Results: 2965 records were screened following exclusion of duplicate citations. Agreement/reliability between the five reviewer pairs for a title-only screening strategy was 89-94% with kappa = 0.54, with 575 records remaining for abstract review. Agreement/reliability for a title-abstract screening strategy was 96-97% with kappa = 0.56, with 257 records remaining for full text review. The final systematic review and meta-analysis included 13 articles, all of which were identified by both screening strategies.
Conclusions: Compared to titles-abstractsstrategy, screening titles-only resulted in a higher number of disagreements but had a similar inter-rater reliability and did not lead to exclusion of any articles used in the final analysis. Initial screening via a titles-only approach may save considerable time in the systematic review process.
Objective: To compare title-abstract and title-only strategies for screening citations for a systematic review.
Methods: A five-member group of physicians performed a systematic review and meta-analysis on light at night and breast cancer risk using Medline and Embase (January to March 2011). Group members were assigned in a round-robin fashion to two reviewer pairs. Each reviewer assessed whether citations included 1) primary data, 2) the exposure of interest, and 3) the outcome of interest. A titles-only screening strategy was undertaken first, followed by a combined titles-abstract strategy on the same citation list. Outcome measures included inter-rater agreement and reliability and articles ultimately included for review following both screening strategies.
Results: 2965 records were screened following exclusion of duplicate citations. Agreement/reliability between the five reviewer pairs for a title-only screening strategy was 89-94% with kappa = 0.54, with 575 records remaining for abstract review. Agreement/reliability for a title-abstract screening strategy was 96-97% with kappa = 0.56, with 257 records remaining for full text review. The final systematic review and meta-analysis included 13 articles, all of which were identified by both screening strategies.
Conclusions: Compared to titles-abstractsstrategy, screening titles-only resulted in a higher number of disagreements but had a similar inter-rater reliability and did not lead to exclusion of any articles used in the final analysis. Initial screening via a titles-only approach may save considerable time in the systematic review process.