Methodology of animal studies and impact on aggregating data: why meta-analyses might be inappropriate for preclinical studies

Article type
Authors
Pecoraro V1, Cappellini G2, Ciccolallo L3, Dall’Olmo L4, Virgili G5, Garattini S6, Moja L7
1Clinical Epidemiology Unit, IRCCS Orthopedic Institute Galeazzi, Milan, Italy
2Chiesi Farmaceutici, Parma, Italy
3European Food Safety Authority, Parma, Italy
4ULSS 15 Padova, Italy
5Department of Oto Specialised Surgical Sciences, University of Florence, Florence, Italy
6Mario Negri Institute of Pharmacological Research, Milan, Italy
7Department of Biomedical Sciences for Health, University of Milan, Milan, Italy
Abstract
Background: For scientific and ethical reasons, experiments involving animals should be appropriately designed, correctly performed and transparently reported.

Objectives: We evaluated the characteristics of reporting, methodology and differences in the efficacy pattern of statins in animal models (mice, rats and rabbits). We explored where these have implications for the conduct and interpretation of meta-analyses.

Methods: We searched Medline and Embase. All eligible articles were examined and results about total cholesterol (TC), myocardial infarction and survival were extracted. We recorded also design and experimental characteristics. Weighted mean difference and odds ratios were pooled. Fixed and random effects models were compared. Heterogeneity, prediction intervals, publication bias, and meta-regression analyses were done.

Results: We included 161 studies and more than 2500 animals. Statins lowered TC in all species considered although with large differences in the magnitude of the effect size: −30% in mice, −20% in rabbits and −10% in rats. Few studies considered strains at high risk of cardiovascular diseases and hard outcomes. Although the majority of studies reported they were randomized (55%), many omitted essential information about gender, age or weight of the animals undermining the opportunity for meta-analyses. In 4% of the studies the number of animals used was not reported. Fixed and random effects models gave different results (ratio of effect size increased by five folds). Within animal models heterogeneity was consistently substantial. Accounting for co-variates had minimal impact on it. Publication bias is highly suspected. Prediction intervals were width.

Conclusions: Although statins showed efficacy in animal models, the preclinical data were scarce, often un-interpretable and difficult to reproduce. Meta-analyses were inconsistent: a reliable approach to estimate the true parameter was imperceptible. In such conditions one would ideally simply consider improper to meta-analyse animal studies.