Article type
Year
Abstract
Introduction: Evidence-Based Health Care [EBHC] is becoming increasingly popular and at least in some settings is being used to assist in the creation of health care policy. Health care providers and consumers need to be confident that the results from meta-analyses are as free as possible from bias [i.e., systematic error]. If meta-analyses are to continue to be useful serious consideration must be given to how they are conducted and reported.
Objectives: This study was carried out to: a) assess the quality of reports and general characteristics of paper-based meta-analyses [PBMA] published between 1990 and 1995; b) assess the quality of reports and general characteristics of meta-analyses over time; c) compare PBMA and Cochrane review for quality of reporting and general characteristics.
Methods: PBMA were randomly selected from a database of systematic reviews. Cochrane reviews were randomly selected from the Cochrane Database of Systematic Reviews. Included studies were masked to title, author, journal and source of support. Data were extracted using a predetermined abstraction form. Two quality assessment instruments were used: a) a checklist developed by Sacks et al. and b) a scale developed by Oxman and Guyatt. The Sacks et al. checklist has 23 items will an ordinal response option [adequate, partial, none or unknown] for each question. The Oxman and Guyatt scale consists of 10 questions, the last one of which asks about the overall scientific quality of the meta-analysis. The scoring range for this question is from 1 to 7 with higher scores, indicating superior quality. Data were analysed using univariate and multivariate approaches. Comparisons were carried out between studies reported in earlier years [i.e., those published in paper-based journal between 1977 and 1989], and later years (i.e., published between 1990 and 1995]. Comparisons were also made between PBMA and Cochrane reviews.
Results: We included 151 articles. Of these, 47 were classified as early years, 52 as later years and 52 as Cochrane reviews. Using the Oxman-Guyatt tool, the mean overall scientific quality of meta-analyses during the early years published was 3.04 [ 95% CI: 2.53, 3.56]. This increased to a mean of 3.35 [95% CI 2.83, 3.87] in the later years. Cochrane reviews scored the highest, mean overall scientific quality 3.42 [95% CI: 2.92, 3.93].
Discussion: This study has revealed that the quality of reports of PBMA has improved over time. However, there is still considerable room for improvement. Cochrane reviews are of similar quality to recently published PBMA. It is possible that the low quality of reports reflects, in part, the lack of evidence-based guidelines for reporting meta-analyses and systematic reviews. The Cochrane Collaboration is starting to achieve this objective through its handbook and iterative peer review process. Paper-based journals face additional obstacles.
Objectives: This study was carried out to: a) assess the quality of reports and general characteristics of paper-based meta-analyses [PBMA] published between 1990 and 1995; b) assess the quality of reports and general characteristics of meta-analyses over time; c) compare PBMA and Cochrane review for quality of reporting and general characteristics.
Methods: PBMA were randomly selected from a database of systematic reviews. Cochrane reviews were randomly selected from the Cochrane Database of Systematic Reviews. Included studies were masked to title, author, journal and source of support. Data were extracted using a predetermined abstraction form. Two quality assessment instruments were used: a) a checklist developed by Sacks et al. and b) a scale developed by Oxman and Guyatt. The Sacks et al. checklist has 23 items will an ordinal response option [adequate, partial, none or unknown] for each question. The Oxman and Guyatt scale consists of 10 questions, the last one of which asks about the overall scientific quality of the meta-analysis. The scoring range for this question is from 1 to 7 with higher scores, indicating superior quality. Data were analysed using univariate and multivariate approaches. Comparisons were carried out between studies reported in earlier years [i.e., those published in paper-based journal between 1977 and 1989], and later years (i.e., published between 1990 and 1995]. Comparisons were also made between PBMA and Cochrane reviews.
Results: We included 151 articles. Of these, 47 were classified as early years, 52 as later years and 52 as Cochrane reviews. Using the Oxman-Guyatt tool, the mean overall scientific quality of meta-analyses during the early years published was 3.04 [ 95% CI: 2.53, 3.56]. This increased to a mean of 3.35 [95% CI 2.83, 3.87] in the later years. Cochrane reviews scored the highest, mean overall scientific quality 3.42 [95% CI: 2.92, 3.93].
Discussion: This study has revealed that the quality of reports of PBMA has improved over time. However, there is still considerable room for improvement. Cochrane reviews are of similar quality to recently published PBMA. It is possible that the low quality of reports reflects, in part, the lack of evidence-based guidelines for reporting meta-analyses and systematic reviews. The Cochrane Collaboration is starting to achieve this objective through its handbook and iterative peer review process. Paper-based journals face additional obstacles.