Background: In meta-analyses as the number of trials in the pooled effect analyses decreases, the risk of false positives or false negatives increases. This is partly due to the assumption of normality that may not hold in small samples. Creation of a distribution from the observed trials using permutation methods to calculate P values may allow for less spurious findings. Permutation has not been empirically tested in meta-regression.

Objectives: The objective of this study was to perform an empirical investigation to explore the differences in results for meta-analyses on a small number of trials using standard large sample approaches verses permutation-based methods for pooled effect estimates.

Methods: We isolated a sample of systematic reviews with varying number of included studies. Finally, we performed meta-analyses on the primary outcome of meta-analysis, collected p-values and confidence intervals. Next we used permutation based resampling to arrive at p-values and bootstrapping to arrive at confidence intervals. We then compared final P values between methods.

Results: We are currently collecting all meta-analyses and will conduct the analyses by June.

Conclusions: We will present empirical data comparing permutation based resampling to standard methods on p-values for pooled effects in meta0analyses. These finding may influence methods.