Article type
Year
Abstract
Background: One of the hallmarks of a well-conducted systematic review (SR) is a comprehensive search. However, empiric evidence is lacking on the impact of database selection on the results of SRs.
Objectives: To examine the impact on the results of SRs based on selective searching.
Methods: The sample was all child-relevant SRs with at least one meta-analysis from three Cochrane Review Groups (Acute Respiratory Infections (ARI), Infectious Diseases, Developmental Psychosocial and Learning Disorders; n = 129). We searched 13 databases to determine the proportion of relevant studies that were indexed in each source. We re-ran the primary meta-analysis with the studies identified in MEDLINE only and in MEDLINE plus each of the other databases. We calculated a ratio of the new point estimate to the original point estimate and a ratio of confidence interval widths. We determined how often results changed in statistical significance.
Results: These results are based on the 57 SRs from ARI; complete results for all SRs will be available in October 2015. The average number of relevant studies was highest for MEDLINE (85%), EMBASE (80%), and BIOSIS (65%). From a total of 398 studies, 65 were not found in MEDLINE; 20 of the 65 were found in EMBASE and 13 in BIOSIS; other databases identified three or less. The mean ratio of point estimates for MEDLINE to the reference standard for dichotomous outcomes (n = 37) was 1.06. The smallest ratio was 1.03 for MEDLINE+BIOSIS. The mean ratio of confidence interval widths of MEDLINE to the reference standard was 1.12; the smallest ratio was 1.09 for MEDLINE+EMBASE. The mean ratio of standardized mean differences (n = 19) was 0.01 for MEDLINE alone, with the mean ratio of confidence interval 1.17 (smallest for MEDLINE+EMBASE, 1.15). For MEDLINE alone, five meta-analyses changed in significance. The fewest number of meta-analyses changing in significance (n = 1) was for MEDLINE+EMBASE.
Conclusions: This study provides quantitative data regarding the impact on SR results of restricting searches to select databases. This information may be useful to increase efficiencies in the conduct of SRs.
Objectives: To examine the impact on the results of SRs based on selective searching.
Methods: The sample was all child-relevant SRs with at least one meta-analysis from three Cochrane Review Groups (Acute Respiratory Infections (ARI), Infectious Diseases, Developmental Psychosocial and Learning Disorders; n = 129). We searched 13 databases to determine the proportion of relevant studies that were indexed in each source. We re-ran the primary meta-analysis with the studies identified in MEDLINE only and in MEDLINE plus each of the other databases. We calculated a ratio of the new point estimate to the original point estimate and a ratio of confidence interval widths. We determined how often results changed in statistical significance.
Results: These results are based on the 57 SRs from ARI; complete results for all SRs will be available in October 2015. The average number of relevant studies was highest for MEDLINE (85%), EMBASE (80%), and BIOSIS (65%). From a total of 398 studies, 65 were not found in MEDLINE; 20 of the 65 were found in EMBASE and 13 in BIOSIS; other databases identified three or less. The mean ratio of point estimates for MEDLINE to the reference standard for dichotomous outcomes (n = 37) was 1.06. The smallest ratio was 1.03 for MEDLINE+BIOSIS. The mean ratio of confidence interval widths of MEDLINE to the reference standard was 1.12; the smallest ratio was 1.09 for MEDLINE+EMBASE. The mean ratio of standardized mean differences (n = 19) was 0.01 for MEDLINE alone, with the mean ratio of confidence interval 1.17 (smallest for MEDLINE+EMBASE, 1.15). For MEDLINE alone, five meta-analyses changed in significance. The fewest number of meta-analyses changing in significance (n = 1) was for MEDLINE+EMBASE.
Conclusions: This study provides quantitative data regarding the impact on SR results of restricting searches to select databases. This information may be useful to increase efficiencies in the conduct of SRs.