Background: Between-study heterogeneity plays an important role in random-effects models for meta-analysis. Most clinical trials are small, and small trials are often associated with larger effect sizes. Possibly there is also a difference in the extent of between-study heterogeneity (τ) between small and large trials.
Objectives: To evaluate empirically whether there is a relationship between trial size and heterogeneity (τ), using meta-analyses from the Cochrane Database of Systematic Reviews from 2009 to 2013.
Methods: We selected the first meta-analysis per intervention review with a dichotomous (n = 2009) or continuous (n = 1254) outcome. The association between estimated τ and trial size was evaluated within meta-analyses using a Bayesian hierarchical model and across meta-analyses using regression. Small trials were predefined as those having standard errors over 0.2 standardized effects.
Results: Most meta-analyses were based on few studies (median 4, Q1 to Q3: 2 to 6), and most (74%) primary studies were small. Within the same meta-analysis, the small-study τ_S2 was larger than the large-study τ_L2: mean ratio 2.11 (95% Credible Interval 1.05 to 3.87) for dichotomous and 3.11 (95% Credible Interval 2.00 to 4.78) for continuous meta-analyses. The imprecision of τ_S was larger than of τ_L: median SE 0.39 versus 0.20 for small-study and large-study meta-analyses with a dichotomous outcome and 0.22 versus 0.13 for those with a continuous outcome. Similar results were found in the across-meta-analyses approach.
Conclusions: Heterogeneity between small studies is larger than between larger studies. The large imprecision with which τ is estimated in a typical small-studies’ meta-analysis is another reason for concern and sensitivity analyses are recommended.