Article type
Year
Abstract
Introduction:
Objectives: Well-conducted systematic reviews are likely to provide the best evidence about the effectiveness of interventions to change provider behaviour. This paper describes the epidemiology, quality and methods of systematic reviews published between 1966 and 1998.
Methods: We searched Medline, Healthstar, The Cochrane Library and contacted experts to identify systematic reviews of interventions to improve professional performance (defined as reviews that report explicit selection criteria). Two independent reviewers assessed all identified papers based on title and abstract against pre- specified inclusion criteria. All potentially relevant reviews were then obtained and assessed independently by two reviewers using a previously validated checklist, including a summary score reflecting the overall quality of the review rated on a 1 (extensive flaws) - 7 (minimal flaws) scale.
Results: 44 systematic reviews fulfilled our inclusion criteria, including 7 Cochrane reviews. 34 reviews were published after 1994. A number of reviews addressed similar problems (eg prevention) or interventions (eg reminders). Common methodological problems included: failure to adequately report criteria for selecting studies to be included in the review, failure to avoid bias in the selection of studies, failure to adequately report criteria used to assess validity and failure to apply criteria to assess the validity of selected studies. Unit of analysis errors were rarely addressed in reviews. 9 reviews undertook quantitative synthesis, although this seemed inappropriate in some. The mean quality score was 4. More recently completed reviews had better quality scores (Spearman rank correlation = 0.463 p<0.001).
Discussion: There has been a dramatic increase in the number of the systematic reviews published over the last 5 years. There was considerable duplication of effort and overlap between reviews in particular areas. The quality of reviews is improving. However, users of such systematic reviews should appraise them carefully before accepting their conclusions. The development of standard methods would improve the quality of such reviews.
Objectives: Well-conducted systematic reviews are likely to provide the best evidence about the effectiveness of interventions to change provider behaviour. This paper describes the epidemiology, quality and methods of systematic reviews published between 1966 and 1998.
Methods: We searched Medline, Healthstar, The Cochrane Library and contacted experts to identify systematic reviews of interventions to improve professional performance (defined as reviews that report explicit selection criteria). Two independent reviewers assessed all identified papers based on title and abstract against pre- specified inclusion criteria. All potentially relevant reviews were then obtained and assessed independently by two reviewers using a previously validated checklist, including a summary score reflecting the overall quality of the review rated on a 1 (extensive flaws) - 7 (minimal flaws) scale.
Results: 44 systematic reviews fulfilled our inclusion criteria, including 7 Cochrane reviews. 34 reviews were published after 1994. A number of reviews addressed similar problems (eg prevention) or interventions (eg reminders). Common methodological problems included: failure to adequately report criteria for selecting studies to be included in the review, failure to avoid bias in the selection of studies, failure to adequately report criteria used to assess validity and failure to apply criteria to assess the validity of selected studies. Unit of analysis errors were rarely addressed in reviews. 9 reviews undertook quantitative synthesis, although this seemed inappropriate in some. The mean quality score was 4. More recently completed reviews had better quality scores (Spearman rank correlation = 0.463 p<0.001).
Discussion: There has been a dramatic increase in the number of the systematic reviews published over the last 5 years. There was considerable duplication of effort and overlap between reviews in particular areas. The quality of reviews is improving. However, users of such systematic reviews should appraise them carefully before accepting their conclusions. The development of standard methods would improve the quality of such reviews.