Background: When the outcome is continuous, the effect size is commonly represented either as a mean difference (MD) or as a standardized mean difference (SMD). When the outcome is measured in different units across trials in the meta-analyses, we have no other choice than to use SMD to combine the outcomes. When the outcome is measured in the same unit in every trial, we can use either MD or SMD. In this latter case, there appears currently to be no agreement about which effect size to prefer. Few quantitative assessments have been conducted with regard to their relative generalizability and statistically power.

Objectives: To empirically examine which index is more generalizable and statistically powerful in meta-analyses when the same unit is used.

Methods: From the Cochrane Database, we included all the meta-analyses in which the continuous outcome was contributed by at least 3 trials. We examined percentage agreement, I-squared statistics and z-scores of MD and SMD in fixed-effect and random-effects models. Generalizability was assessed as percentage agreement, when one study was taken from each meta-analysis and MD and SMD of that individual trial was compared with the meta-analytically pooled MD and the SMD of the remaining trials. The agreement was defined when the point estimate of MD or SMD of the individual trial is included within the 95% confidence interval of the pooled MD or SMD of the remaining trials. This procedure was repeated for all the trials, and the overall percentage agreement was calculated. I-squared statistics, which index heterogeneity among the combined trials, relate to the generalizability, and z-scores represent the statistically power.

Results: We are currently conducting the analysis. We will present the results at the colloquium.