Teaching searching in an intensive systematic review course: ‘how many citations should I expect to review?’

Article type
Authors
Rosman L1, Twose C1, Li M1, Li T1, Saldanha I1, Dickersin K1
1US Cochrane Center; Cochrane Eyes and Vision Group US Project
Abstract
Background: We offer an intensive 8 week systematic review course in a School of Public Health. The course is unique, requiring student groups to complete all steps of a systematic review on intervention effectiveness (randomized controlled trials, RCTs) or etiology (observational studies, OBS). Informationists oversee searching techniques. One challenge is teaching students to create a high quality search that produces a manageable number of results; only 3 weeks is allocated for searching and title abstract review. Although there has been research on the number of databases needed to search, data are sparse on the number of citations needed to review for systematic reviews of RCTs and OBS.

Objectives: To calculate number needed to be reviewed and read for RCT and OBS reviews.

Methods: Using 2 years of data from 19 groups we abstracted data on total number of citations and full text reports retrieved and reviewed, databases from which they were identified, and number ultimately included. We analyzed frequency statistics overall (mean, median, range) and by database and type of review. We investigated whether included studies were in PubMed, EMBASE, or CENTRAL.

Results: The 19 systematic reviews retrieved on average 4464 unique citations ranging from 1060 to 16 747 (median: 3720). Students retrieved and reviewed on average 183 full text reports (range: 31–357) and abstracted data from on average 23 articles (range: 19–27). Thus, about 200 citations and 5 full text reports need to be reviewed to include 1 study in data abstraction. Results were generally similar for RCTs and OBS. Almost all included studies from 11 reviews analyzed to date are indexed in EMBASE.

Conclusions: The wide range of citations reviewed reveals a discrepancy in workload between student groups. We plan to investigate factors contributing to this wide range and the surprising similarity of results for RCT and OBS reviews.