Article type
Year
Abstract
Background: Some evidence on medical interventions may come from old data. We describe how old data are in meta-analyses of medical interventions and whether reviewers probe the relevance of old data for current practice.
Objectives: We aimed to describe how recent the synthesized evidence is in a representative random sample of the Cochrane Database of Systematic Reviews (CDSR). We also aimed to evaluate whether systematic reviewers discussed how old the trials were and what the implications might be from the inclusion of old trials. A sample of meta-analyses published recently in high-profile medical journals was also analyzed.
Methods: We assessed a 10% random sample of the Cochrane Database of Systematic Reviews (CDRS) (issue 4, 2005) with 1 meta-analysis. We focused from each review on the primary outcome meta-analysis. We assessed the time since publication of the included trials, whether inferences changed if data were limited to trials published in the last decade, and implications of old data were discussed.
Results: For 157 meta-analyses (n=1,149 trials), the median year of last search was 2003 (interquartile range [IQR], 2002-2004). Most meta-analyses (64%) had no studies published in 2001-5. Forty-seven meta-analyses had no data published in the last decade, and in another 16 the statistical significance of the summary effects would change, when data were limited to trials published in the last decade. Only 12 (8%) systematic reviews discussed potential implications of including old studies. We also assessed 22 meta-analyses published in high-impact medical journals in 2005. Only two meta-analyses had no studies published in 2001-2005, while 18 meta-analyses included at least one study published before 1996. Only four meta-analyses discussed implications of including old studies.
Conclusions: For many healthcare interventions evidence from very recent studies is scant. Systematic reviewers and clinicians should be sensitized to the need to discuss the potential implications of some evidence being old.
Objectives: We aimed to describe how recent the synthesized evidence is in a representative random sample of the Cochrane Database of Systematic Reviews (CDSR). We also aimed to evaluate whether systematic reviewers discussed how old the trials were and what the implications might be from the inclusion of old trials. A sample of meta-analyses published recently in high-profile medical journals was also analyzed.
Methods: We assessed a 10% random sample of the Cochrane Database of Systematic Reviews (CDRS) (issue 4, 2005) with 1 meta-analysis. We focused from each review on the primary outcome meta-analysis. We assessed the time since publication of the included trials, whether inferences changed if data were limited to trials published in the last decade, and implications of old data were discussed.
Results: For 157 meta-analyses (n=1,149 trials), the median year of last search was 2003 (interquartile range [IQR], 2002-2004). Most meta-analyses (64%) had no studies published in 2001-5. Forty-seven meta-analyses had no data published in the last decade, and in another 16 the statistical significance of the summary effects would change, when data were limited to trials published in the last decade. Only 12 (8%) systematic reviews discussed potential implications of including old studies. We also assessed 22 meta-analyses published in high-impact medical journals in 2005. Only two meta-analyses had no studies published in 2001-2005, while 18 meta-analyses included at least one study published before 1996. Only four meta-analyses discussed implications of including old studies.
Conclusions: For many healthcare interventions evidence from very recent studies is scant. Systematic reviewers and clinicians should be sensitized to the need to discuss the potential implications of some evidence being old.