Introduction/Objective: Due to the importance of evidence-based health care and the explosion of information in the literature, meta-analyses are becoming increasingly prevalent. Efforts to develop guidelines for meta analyses have been proposed, but have not to date been evaluated.

Methods: To assess the quality of meta-analyses, we examined a case series of meta-analyses submitted to a weekly medical journal during 1996-97.

Results: Of 118 meta-analyses submitted, we calculated percentages, demonstrating acceptable ratings on several characteristics. More than 90% of the manuscripts stated hypotheses and appropriate conclusions, provided the basis for coding data, considered alternative explanations for results, stated explicit inclusion and exclusion criteria, and used appropriate statistical analyses. More than half reported a test for heterogeneity, provided guidelines for future research, documented data classification and coding, assessed publication bias, and permitted replication of results. Less than 1/3 of the manuscripts assessed studied quality, or assessed comparability of cases and controls. Fewer than half (49%) of the manuscripts used only randomized controlled trials (RCTs). Low ratings on the characteristics; were associated with meta-analyses using observational studies.

Discussion: Meta-analyses submitted to a weekly medical journal meet most methodologic standards. Lower percentages of manuscripts assessed study quality, publication bias, and comparability of cases and controls, as well as provided detail to enable replication. These problems were more pronounced in meta-analyses using data from observational studies. This study documents the current status of the quality of submitted meta-analyses and indicates directions for future research and publication guidelines.