Background: Most primary studies included in Cochrane reviews report analyses based on normal distributions and homoscedasticity. When these assumptions are not met, the primary studies report wrong effect sizes and confidence intervals. Meta-analytic results of these primary studies will also be wrong. Meta-analytic methods also assume normal distributions and homoscedasticity, causing the meta-analytic results to be even more biased than the primary studies they are based on. Robust methods are available that yield correct estimates of effect size and confidence intervals, but few researchers know of them.

Objectives: To re-analyze a selection of meta-analyses in Cochrane reviews using robust methods and compare the robust results with the original results.

Methods: We assessed all Cochrane reviews published by the Cochrane Musculoskeletal Review Group (n = 262) and included reviews with meta-analyses of more than 5 studies. If a review had more than one meta-analysis, we included the first reported one. We categorized each analysis in the following manner:1. the entire confidence interval included a negative effect2. the confidence interval crossed the null effect line3. the entire confidence interval included a positive effect. We estimated the proportion of meta-analyses where the original and the re-analyzed results were in separate categories. We also estimated the proportion where the results had shifted from 1. to 3. or from 3. to 1.

Results: The results for the re-analyzes will be presented at the Colloquium.

Conclusions: The Cochrane Collaboration should heighten awareness of bias in effect sizes and confidence intervals caused by skewed distributions and outliers. This applies both to included primary studies and to meta-analyses in reviews.