Article type
Year
Abstract
Background: Systematic reviews and meta-analyses published in leading journals often have important methodological limitations. For example, they may not be based on a comprehensive literature search and the quality of included trials may not have been assessed.
Objectives: To assess the methodology of meta-analyses published in leading general and specialist medical journals over a ten-year period.
Methods: Volumes 1993 to 2002 of four general medicine journals (Annals of Internal Medicine, BMJ, Lancet, JAMA) and four specialist journals (Cancer, Cardiology, Circulation, Obstetrics & Gynecology) were searched by hand for meta-analyses including at least five controlled trials with binary endpoints. Characteristics were assessed using a standardised questionnaire. Trends over time were examined for two-year periods from 1993/4 to 2001/2.
Results: Two hundred and seventy-two meta-analyses, which included a median of 11 trials (range 5-195), were assessed. Most (81%) were published in general medicine journals. Efforts to identify relevant trials intensified over time. For example, the median (range) number of databases searched increased from 1 (1-9) in 1993/4 to 3.5 (1-21) in 2001/2, p<0.0001. The proportion of meta-analyses including searches by hand (10% in 1993/4, 25% in 2001/2, p=0.005), searches of the grey literature (29%, 51%, p=0.010) and of trial registers (10%, 32%, p=0.025) also increased, whereas language restrictions became less common (21%, 5%, p=0.11). Assessments of the quality of trials became more common (45%, 70%, p=0.008), including whether allocation of patients to treatment groups had been concealed (24%, 60%, p=0.001). The use of tests of heterogeneity (76%, 93%, p=0.016) and funnel plots (5%, 47%, p<0.0001) also increased. Improvements were more pronounced following the publication of the QUOROM (QUality Of Reporting Of Meta-analyses) Statement, and less pronounced in general medicine compared to specialist journals.
Conclusions: The quality of meta-analyses published in leading journals has improved in recent years but considerable room for improvement remains.
Objectives: To assess the methodology of meta-analyses published in leading general and specialist medical journals over a ten-year period.
Methods: Volumes 1993 to 2002 of four general medicine journals (Annals of Internal Medicine, BMJ, Lancet, JAMA) and four specialist journals (Cancer, Cardiology, Circulation, Obstetrics & Gynecology) were searched by hand for meta-analyses including at least five controlled trials with binary endpoints. Characteristics were assessed using a standardised questionnaire. Trends over time were examined for two-year periods from 1993/4 to 2001/2.
Results: Two hundred and seventy-two meta-analyses, which included a median of 11 trials (range 5-195), were assessed. Most (81%) were published in general medicine journals. Efforts to identify relevant trials intensified over time. For example, the median (range) number of databases searched increased from 1 (1-9) in 1993/4 to 3.5 (1-21) in 2001/2, p<0.0001. The proportion of meta-analyses including searches by hand (10% in 1993/4, 25% in 2001/2, p=0.005), searches of the grey literature (29%, 51%, p=0.010) and of trial registers (10%, 32%, p=0.025) also increased, whereas language restrictions became less common (21%, 5%, p=0.11). Assessments of the quality of trials became more common (45%, 70%, p=0.008), including whether allocation of patients to treatment groups had been concealed (24%, 60%, p=0.001). The use of tests of heterogeneity (76%, 93%, p=0.016) and funnel plots (5%, 47%, p<0.0001) also increased. Improvements were more pronounced following the publication of the QUOROM (QUality Of Reporting Of Meta-analyses) Statement, and less pronounced in general medicine compared to specialist journals.
Conclusions: The quality of meta-analyses published in leading journals has improved in recent years but considerable room for improvement remains.