Overestimation of treatment effects associated with small sample size in osteoarthritis research

Article type
Authors
Jüni P, Nüesch E, Reichenbach S, Rutjes A, Scherrer M, Bürgi E, Trelle S
Abstract
Background: The impact of small trials on meta-analyses of continuous outcomes has not been systematically assessed. Objectives: We determined whether small sample size is associated with an overestimation of treatment effects in osteoarthritis research. Methods: We searched for meta-analyses of at least two randomized controlled trials comparing an intervention with placebo or no intervention in patients with hip or knee osteoarthritis. We calculated effect sizes (ES) from between-group differences in means of pain intensity divided by the pooled standard deviation. Within each meta-analysis, we calculated the difference in ES between large trials with at least 200 patients and smaller trials. Then, we determined the impact of restricting random-effects meta-analyses to large trials. Results: Thirteen meta-analyses including 156 trials and 37,594 patients contributed. Seventy-one trials had included at least 200 patients, 85 trials were smaller. Large trials were more likely than small trials to report adequate concealment of allocation, to be described as double-blind, to have an intention-to-treat analysis, and to report the primary outcome and power calculations. Small trials showed more beneficial results than large trials (difference in ES, -0.23, 95%-CI -0.37 to -0.09). Differences between small and large trials were more evident in meta-analyses with large between-trial heterogeneity (difference in ES, -0.43, 95%-CI -0.66 to -0.20), as compared with meta-analyses with small to moderate heterogeneity (-0.10, 95%-CI -0.21 to 0.01, p-value for interaction, 0.028). Treatment effects became less beneficial in ten meta-analyses and more beneficial in three after a restriction to large trials (range 0.52 ES less to 0.08 ES more beneficial). Results became less heterogeneous in six meta-analyses and more heterogenous in seven. In three meta-analyses, results lost statistical significance at the 5% level. Conclusions: In osteoarthritis research, the overestimation of treatment effects associated with small sample size is particularly prominent in metaanalyses with a high degree of between-trial heterogeneity. Potential mechanisms include a combination of inadequate methodology and reporting biases.