Article type
Year
Abstract
Background: The need for more scientifically validated search strategies is increasing. To date, research on how best to identify trials within geographic regions is limited. In the absence of a single electronic database providing comprehensive coverage of African trials, identifying African-specific publications in bibliographic databases such as MEDLINE is problematic because of inconsistencies in indexing, and incomplete reporting by authors, especially regarding trial location. We therefore developed a comprehensive African filter for identifying trials conducted in Africa1.
Objectives: To test the sensitivity and precision of an African search filter in identifying randomized controlled trials (RCTs) conducted in Africa using PubMed/MEDLINE.
Methods: The African filter comprised country and regional names, using MeSH terms and text words. The filter was combined with the PubMed version of the Cochrane Highly Sensitive Search Strategy for reports of RCTs and the comprehensive HIV/AIDS strategy developed by the Cochrane HIV/AIDS Review Group. We initially searched PubMed/MEDLINE for all HIV/AIDS RCTS, irrespective of location, published in 2004. Two investigators (LG and KB) independently handsearched the retrieved citations and abstracts to identify HIV/AIDS RCTs conducted in Africa, forming a "gold standard" reference set. A second PubMed/MEDLINE search was conducted on the same day using the African filter in combination with the initial search strategy. The retrieved references were handsearched to identify HIV/AIDS RCTs conducted in Africa. Any uncertainty was resolved by another investigator (NS). We compared the records found by the African filter with the "gold standard" reference set. The sensitivity of the African filter was calculated by dividing the number of RCTs correctly found using the filter by the number of RCTs identified in the "gold standard" reference set, expressed as a percentage. The precision of each search strategy was calculated as the percentage of relevant RCTs divided by the total number of records retrieved by that strategy. We identified reasons for non-retrieval of African trials by the filter by scrutinising individual records.
Results: The "gold standard" reference set comprised 1799 records with 20 of these identified as HIV/AIDS RCTs conducted in Africa. The precision was 1.1%. The African filter in combination with the comprehensive HIV/AIDS search strategy identified 180 records, of which 17 were African RCTs. The sensitivity of the African filter search was 85% and the precision was 9.4%. The African filter failed to identify three RCTs conducted in Africa. One trial was identified as African based on prior knowledge as there was no mention of trial location in the record. The remaining two trials were identified as African based on information provided in the Address field ([AD]). Further investigation of the filter revealed that "country name" followed by"[tw]" (text word) failed to search the address field of records. The two trials were correctly identified by the search strategy when the syntax "[tw]" was replaced with an asterisk (*) after the country name, ensuring that the term searched across "all fields" , including the address field, improving sensitivity to 95%, with minimal loss of precision.
Conclusions: The addition of the African filter to the comprehensive HIV/AIDS RCT search strategy improved the precision of the strategy tenfold and reduced the number of records to be searched by 90%. Sensitivity was high at 85% with a slight modification of the search syntax improving it to 95%. Our findings show that our comprehensive geographic filter significantly reduced workload while maintaining sensitivity. Further research is required to assess the generalisability of our findings to other geographic filters.
Reference:
1. Siegfried N, Clarke M, Volmink J. (2005). Randomized controlled trials in Africa of HIV and AIDS: descriptive study and spatial distribution. BMJ 331: 742-6.
Objectives: To test the sensitivity and precision of an African search filter in identifying randomized controlled trials (RCTs) conducted in Africa using PubMed/MEDLINE.
Methods: The African filter comprised country and regional names, using MeSH terms and text words. The filter was combined with the PubMed version of the Cochrane Highly Sensitive Search Strategy for reports of RCTs and the comprehensive HIV/AIDS strategy developed by the Cochrane HIV/AIDS Review Group. We initially searched PubMed/MEDLINE for all HIV/AIDS RCTS, irrespective of location, published in 2004. Two investigators (LG and KB) independently handsearched the retrieved citations and abstracts to identify HIV/AIDS RCTs conducted in Africa, forming a "gold standard" reference set. A second PubMed/MEDLINE search was conducted on the same day using the African filter in combination with the initial search strategy. The retrieved references were handsearched to identify HIV/AIDS RCTs conducted in Africa. Any uncertainty was resolved by another investigator (NS). We compared the records found by the African filter with the "gold standard" reference set. The sensitivity of the African filter was calculated by dividing the number of RCTs correctly found using the filter by the number of RCTs identified in the "gold standard" reference set, expressed as a percentage. The precision of each search strategy was calculated as the percentage of relevant RCTs divided by the total number of records retrieved by that strategy. We identified reasons for non-retrieval of African trials by the filter by scrutinising individual records.
Results: The "gold standard" reference set comprised 1799 records with 20 of these identified as HIV/AIDS RCTs conducted in Africa. The precision was 1.1%. The African filter in combination with the comprehensive HIV/AIDS search strategy identified 180 records, of which 17 were African RCTs. The sensitivity of the African filter search was 85% and the precision was 9.4%. The African filter failed to identify three RCTs conducted in Africa. One trial was identified as African based on prior knowledge as there was no mention of trial location in the record. The remaining two trials were identified as African based on information provided in the Address field ([AD]). Further investigation of the filter revealed that "country name" followed by"[tw]" (text word) failed to search the address field of records. The two trials were correctly identified by the search strategy when the syntax "[tw]" was replaced with an asterisk (*) after the country name, ensuring that the term searched across "all fields" , including the address field, improving sensitivity to 95%, with minimal loss of precision.
Conclusions: The addition of the African filter to the comprehensive HIV/AIDS RCT search strategy improved the precision of the strategy tenfold and reduced the number of records to be searched by 90%. Sensitivity was high at 85% with a slight modification of the search syntax improving it to 95%. Our findings show that our comprehensive geographic filter significantly reduced workload while maintaining sensitivity. Further research is required to assess the generalisability of our findings to other geographic filters.
Reference:
1. Siegfried N, Clarke M, Volmink J. (2005). Randomized controlled trials in Africa of HIV and AIDS: descriptive study and spatial distribution. BMJ 331: 742-6.