Challenges of developing a geographic search filter to identify randomized controlled trials in Africa: finding the optimal balance between sensitivity and precision

Article type
Authors
Pienaar E, Grobler L, Siegfried N, Eisinga A
Abstract
Background: Research on how to best identify trials within geographic regions is limited. Identifying African-specific publications in bibliographic databases is problematic because of indexing inconsistencies and incomplete reporting by authors, especially regarding trial location. We developed a comprehensive African filter for identifying trials conducted in Africa. Objectives: (1) To test sensitivity and precision of African search filter in identifying randomized controlled trials (RCTs) conducted in Africa using EMBASE; and (2) to compare the results using the filter in PubMed/MEDLINE. Methods: The African filter comprised country and regional names, combined with terms from The Cochrane Collaboration’s annual EMBASE systematic search for RCT reports and the Cochrane HIV/AIDS strategy. We searched EMBASE for HIV/AIDS RCTS, irrespective of location, published in 2004. Two investigators independently handsearched retrieved citations and abstracts to identify African HIV/AIDS RCTs, forming a ‘gold standard’ reference set. A second EMBASE search was conducted on the same day using the African filter combined with the initial search strategy. Retrieved references were handsearched to identify African HIV/AIDS RCTs. We compared the African filter reference set with the ‘gold standard’ reference set. Reasons for non-retrieval of African trials were identified by scrutinizing the full text. Results: The ‘gold standard’ reference set comprised 763 records, of which 37 were African HIV/AIDS RCTs. Precision was 5%. The African filter set comprised 98 records, of which 27 were African RCTs. Sensitivity of the African filter search was 73% and precision was 28%. The African filter failed to identify ten African RCTs. Six were identified through author affiliation, one through prior knowledge and three through full-text. Table 1 shows the sensitivity and precision of the African filter in EMBASE and PubMed/MEDLINE. The sensitivity of the filter was similar for both databases, with precision somewhat higher in EMBASE than in PubMed/MEDLINE. Conclusions: Adding the African filter to the comprehensive HIV/AIDS RCT search strategy improved the precision of the strategy in EMBASE and PubMed/MEDLINE. Our findings show that our comprehensive geographic filter significantly reduced workload in both databases with, however, a loss in sensitivity.