Article type
Abstract
Background: Standard random-effects meta-analysis is widely used for synthesizing studies in Cochrane Reviews. The key assumption of this model is that the effects underlying the studies in the meta-analysis come from a common normal distribution. However, the presence of large heterogeneity or outlying studies may violate this assumption and invalidate the results.
Objectives: To identify and compare alternative flexible models for synthesizing results from heterogeneous studies.
Methods: We first conducted a methodological systematic review to identify articles that proposed meta-analysis models relaxing the strong assumption of a single normal distribution for the random effects. Subsequently, we performed a simulation study to evaluate the performance of the identified models and to compare them with the standard normal model. We considered 22 scenarios varying the amount of heterogeneity between studies, the shape of the true distribution, and the number of included studies. We also applied the different models to an exemplar meta-analysis of 23 heterogeneous observational studies assessing the relationship between metabolic syndrome and psoriasis.
Results: We identified 1022 articles in PubMed and further added relevant articles through hand-searching in Google Scholar and other related journals. We ended up with 13 eligible articles suggesting 10 alternative meta-analysis models. Simulations revealed that the bias of both the mean treatment effect and heterogeneity is substantial in presence of high heterogeneity regardless of the model used. However, more flexible models can better estimate the underlying effects of the studies and capture their true distribution when the latter is not normal. In the real example, the use of certain alternative models not only captured better the heterogeneity among studies but also allowed us to automatically identify subgroups of studies with similar characteristics, such as homogeneous populations or effect estimates adjusted for the same confounding factors.
Conclusion: When substantial heterogeneity is suspected or outlying studies are present, focusing on the mean summary effect may lead to spurious conclusions. In such cases, the relevance of the standard random-effects meta-analysis should be assessed thoroughly, and alternative synthesis models describing better the variation of the underlying effects of the studies should be used.
Objectives: To identify and compare alternative flexible models for synthesizing results from heterogeneous studies.
Methods: We first conducted a methodological systematic review to identify articles that proposed meta-analysis models relaxing the strong assumption of a single normal distribution for the random effects. Subsequently, we performed a simulation study to evaluate the performance of the identified models and to compare them with the standard normal model. We considered 22 scenarios varying the amount of heterogeneity between studies, the shape of the true distribution, and the number of included studies. We also applied the different models to an exemplar meta-analysis of 23 heterogeneous observational studies assessing the relationship between metabolic syndrome and psoriasis.
Results: We identified 1022 articles in PubMed and further added relevant articles through hand-searching in Google Scholar and other related journals. We ended up with 13 eligible articles suggesting 10 alternative meta-analysis models. Simulations revealed that the bias of both the mean treatment effect and heterogeneity is substantial in presence of high heterogeneity regardless of the model used. However, more flexible models can better estimate the underlying effects of the studies and capture their true distribution when the latter is not normal. In the real example, the use of certain alternative models not only captured better the heterogeneity among studies but also allowed us to automatically identify subgroups of studies with similar characteristics, such as homogeneous populations or effect estimates adjusted for the same confounding factors.
Conclusion: When substantial heterogeneity is suspected or outlying studies are present, focusing on the mean summary effect may lead to spurious conclusions. In such cases, the relevance of the standard random-effects meta-analysis should be assessed thoroughly, and alternative synthesis models describing better the variation of the underlying effects of the studies should be used.