Article type
Year
Abstract
Background: Systematic reviews are becoming a very important tool to summarize clinical evidence, and are used to develop clinical recommendations. Systematic reviews of high quality assure a good level of confidence on the strength of the recommendations. It is therefore important that reviews be of high quality. Until now no agreement has been reached on a common approach to evaluate systematic reviews.
Objectives: To assess the quality of the sources used for the implementation of guidelines on prevention and treatment of breast and colorectal cancers.
Methods: A QUOROM-based checklist was developed and applied to the reviews cited as source of information in a representative and recent sample of guidelines. The QUOROM-based checklist was provided a weight for each criterion and a total quality score. Each review was independently evaluated by two reviewers; disagreements were solved by consensus.
Results: Out of 83 identified reviews, 80 (96.4%) were retrieved and evaluated; 36 focused on breast, 44 on colorectal cancer. Twenty-three reviews (28.7%) did not match the definition of systematic review. In 7 reviews (8.7%) the searching method was described elsewhere; in 10 reviews (12.5%) was unclear. Forty reviews (50%) were clearly systematic. Not systematic, low and very low quality reviews accounted for 70% of the total, respectively 80.6% of breast and 61.4% of colorectal cancer reviews. No reviews reached the A+ quality class (score 46-50), and only 5 (6.2%) reached the A- class (41-45); 7 (8.7%) obtained the score of B+ class (36-40). Only 30.3% of breast and 26.1% of colorectal cancer guidelines cited in bibliography at least one review of good quality. Twenty-one percent of breast and 52.1% of colorectal cancer guidelines used as source of information only not systematic reviews or no review at all.
Conclusions: The results of this assessment provide a sober picture of the overall quality of the sources used to build guidelines. When using recommendations oncologists should be aware that they could be relying on poor underlying documents. When writing recommendations, writing groups should be aware of the methodological problems involved, and are warmly invited to consult the existing manuals for the preparation of practice guidelines.
Objectives: To assess the quality of the sources used for the implementation of guidelines on prevention and treatment of breast and colorectal cancers.
Methods: A QUOROM-based checklist was developed and applied to the reviews cited as source of information in a representative and recent sample of guidelines. The QUOROM-based checklist was provided a weight for each criterion and a total quality score. Each review was independently evaluated by two reviewers; disagreements were solved by consensus.
Results: Out of 83 identified reviews, 80 (96.4%) were retrieved and evaluated; 36 focused on breast, 44 on colorectal cancer. Twenty-three reviews (28.7%) did not match the definition of systematic review. In 7 reviews (8.7%) the searching method was described elsewhere; in 10 reviews (12.5%) was unclear. Forty reviews (50%) were clearly systematic. Not systematic, low and very low quality reviews accounted for 70% of the total, respectively 80.6% of breast and 61.4% of colorectal cancer reviews. No reviews reached the A+ quality class (score 46-50), and only 5 (6.2%) reached the A- class (41-45); 7 (8.7%) obtained the score of B+ class (36-40). Only 30.3% of breast and 26.1% of colorectal cancer guidelines cited in bibliography at least one review of good quality. Twenty-one percent of breast and 52.1% of colorectal cancer guidelines used as source of information only not systematic reviews or no review at all.
Conclusions: The results of this assessment provide a sober picture of the overall quality of the sources used to build guidelines. When using recommendations oncologists should be aware that they could be relying on poor underlying documents. When writing recommendations, writing groups should be aware of the methodological problems involved, and are warmly invited to consult the existing manuals for the preparation of practice guidelines.