Article type
Year
Abstract
Introduction: Several recent large randomized control trials have not supported the effectiveness of treatment found in meta-analyses of smaller trials.
Objective: To compare and evaluate the results of meta-analyses of "smaller" studies with the "largest" study or studies.
Methods: We collected over 40 published meta-analyses that cover a variety of treatments for different outcomes. A large study is defined as having at least 1,000 patients or meeting statistical power considerations. For each of these two definitions, results of the large studies (or meta-analyses of large studies) are compared with meta-analyses of smaller studies with respect to statistical significance, whether the relative risk estimate of each lies in the other's 95% confidence interval, and difference in Z values. A random effects model of relative risk is used.
Results: All comparisons are consistent or can be explained by differences in protocols, control rates, or sample sizes (random error). In a plot of relative risk estimates and 95% confidence intervals against differences in Z values (DZ), these comparisons can be categorized into three clusters; |DZ| ? 1; DZ >1; DZ, -1. For rtPA versus streptokinase, for example, DZ = -1.51 -1.51 indicates that the protocol differed between the large studies (GISSI-2, ISG, ISIS-3, GUSTO) and the meta-analysis of smaller studies. Similarly in the case of intravenous magnesium for acute myocardial infarction (DZ = -2.76), where the largest study (ISIS-4) gave intravenous magnesium differently, which other data suggest would be ineffective, and had a significantly lower control rate.
Discussion: Meta-analyses of smaller studies are generally comparable with results from large studies, but meta-regression is needed to account for factors such as duration of follow-up, treatment delivery, and control rates that may be associated with treatment effect.
Objective: To compare and evaluate the results of meta-analyses of "smaller" studies with the "largest" study or studies.
Methods: We collected over 40 published meta-analyses that cover a variety of treatments for different outcomes. A large study is defined as having at least 1,000 patients or meeting statistical power considerations. For each of these two definitions, results of the large studies (or meta-analyses of large studies) are compared with meta-analyses of smaller studies with respect to statistical significance, whether the relative risk estimate of each lies in the other's 95% confidence interval, and difference in Z values. A random effects model of relative risk is used.
Results: All comparisons are consistent or can be explained by differences in protocols, control rates, or sample sizes (random error). In a plot of relative risk estimates and 95% confidence intervals against differences in Z values (DZ), these comparisons can be categorized into three clusters; |DZ| ? 1; DZ >1; DZ, -1. For rtPA versus streptokinase, for example, DZ = -1.51 -1.51 indicates that the protocol differed between the large studies (GISSI-2, ISG, ISIS-3, GUSTO) and the meta-analysis of smaller studies. Similarly in the case of intravenous magnesium for acute myocardial infarction (DZ = -2.76), where the largest study (ISIS-4) gave intravenous magnesium differently, which other data suggest would be ineffective, and had a significantly lower control rate.
Discussion: Meta-analyses of smaller studies are generally comparable with results from large studies, but meta-regression is needed to account for factors such as duration of follow-up, treatment delivery, and control rates that may be associated with treatment effect.