Article type
Year
Abstract
Background: Evidence from multiple randomized controlled trials (RCTs) accumulates over many years and waits for meta-analysis to provide ‘conclusive’ evidence of treatment effect. This practice may delay the implementation of potentially beneficial treatments. Additionally, even after sufficient evidence has built up, RCTs are still conducted, potentially wasting resources or introducing harm.
Objectives: To examine how often the result of the first trial, last trial or any trial selected at random agree with the corresponding meta-analysis.
Methods: Published meta-analyses were identified from five key medical journals and also from the Cochrane Database of Systematic Reviews. Meta-analyses were included where an intervention was compared with placebo, no intervention or usual care. The effect size of the earliest, most recent or any trial in the meta-analysis was compared with the meta-analysis effect size.
Results: A total of 647 meta-analyses were included; the median number of included studies was 5.2. In 36% (n = 233) of meta-analyses, the earliest published study reported a significant result and the corresponding meta-analysis was significant and in the same direction 84.1% (95% confidence interval (CI) 79.4 to 88.8) of the time. When the earliest study did not have a significant result, the meta-analysis was non-significant in 57.9% (95% CI 53.2 to 62.8) of cases. Similar results were observed when the last or any trial, were substituted in analyses. No instances were identified where the earliest trial and meta-analysis were both significant but in the opposite direction. In meta-analyses with five or more studies, the median time from first to fifth study was 6.5 years.
Conclusions: The first trial (or any trial to hand) demonstrating clinical effectiveness is likely to reflect the true treatment effect. This finding has important implications in urgent or critical circumstances where no other effective treatment is available. Cautious recommendations, depending on circumstances or potential harms, may be made if the first trial demonstrates effectiveness. Effective treatment could be implemented many years in advance of a meta-analysis.
Objectives: To examine how often the result of the first trial, last trial or any trial selected at random agree with the corresponding meta-analysis.
Methods: Published meta-analyses were identified from five key medical journals and also from the Cochrane Database of Systematic Reviews. Meta-analyses were included where an intervention was compared with placebo, no intervention or usual care. The effect size of the earliest, most recent or any trial in the meta-analysis was compared with the meta-analysis effect size.
Results: A total of 647 meta-analyses were included; the median number of included studies was 5.2. In 36% (n = 233) of meta-analyses, the earliest published study reported a significant result and the corresponding meta-analysis was significant and in the same direction 84.1% (95% confidence interval (CI) 79.4 to 88.8) of the time. When the earliest study did not have a significant result, the meta-analysis was non-significant in 57.9% (95% CI 53.2 to 62.8) of cases. Similar results were observed when the last or any trial, were substituted in analyses. No instances were identified where the earliest trial and meta-analysis were both significant but in the opposite direction. In meta-analyses with five or more studies, the median time from first to fifth study was 6.5 years.
Conclusions: The first trial (or any trial to hand) demonstrating clinical effectiveness is likely to reflect the true treatment effect. This finding has important implications in urgent or critical circumstances where no other effective treatment is available. Cautious recommendations, depending on circumstances or potential harms, may be made if the first trial demonstrates effectiveness. Effective treatment could be implemented many years in advance of a meta-analysis.