Article type
Year
Abstract
Background: Systematic reviews of prognosis are important for informing healthcare. Assessing the quality of included studies is fundamental, however there is limited consensus on how to proceed.
Objectives: We reviewed and appraised quality assessment in systematic reviews of prognostic studies. We developed recommendations for quality appraisal in future systematic reviews of prognosis and assessed the clinimetric properties of this new approach.
Methods: Medline was searched and cited references were reviewed, identifying 757 potential systematic reviews of prognosis studies. 71 systematic reviews that assessed methodological quality of included studies were included. Characteristics of the reviews were extracted as well as detailed information on the quality assessment. Guidelines for quality appraisal in systematic reviews of prognosis were developed. To assess the properties of this new approach, a composite tool including items from our recommendations and other common criteria lists is tested in two ongoing systematic electronic reviews of prognosis. For each review, two sets of raters evaluate quality in a minimum of 30 primary studies. Items for each QA tool are extracted and graded according to the developers. Analyses include sensibility, validity, inter-rater reliability within tools and agreement between tools, and agreement with a judgment of overall study quality.
Results: We constructed a list of 378 distinct quality items and grouped them into 24 domains/ideas. Fourteen domains mapped onto a theoretical framework addressing potential opportunities for bias including: 1. study participation, 2. study attrition, 3. prognostic factor measurement, 4. confounding measurement and account, 5. outcome measurement, and 6. analysis. Adequate assessment of each potential biases ranged from 12.7% (for confounding measurement and account) to 77.5% (for prognostic factor measurement). Only two reviews assessed all parts of the theoretical model. We make recommendations to improve the quality assessment of future systematic reviews. Clinimetric testing results will be presented at the Cochrane Colloquium.
Conclusions: Systematic reviews are frequently used to summarize information on health care topics and guide policy. In many reviews of prognosis quality appraisal is inadequate and done with little standardization, calling into question the scientific validity of the results. We recommend quality appraisal focusing on key potential biases in reviews of prognosis. A transparent approach will help to improve this literature and facilitate future study on the importance of potential biases in the study of prognosis.
Objectives: We reviewed and appraised quality assessment in systematic reviews of prognostic studies. We developed recommendations for quality appraisal in future systematic reviews of prognosis and assessed the clinimetric properties of this new approach.
Methods: Medline was searched and cited references were reviewed, identifying 757 potential systematic reviews of prognosis studies. 71 systematic reviews that assessed methodological quality of included studies were included. Characteristics of the reviews were extracted as well as detailed information on the quality assessment. Guidelines for quality appraisal in systematic reviews of prognosis were developed. To assess the properties of this new approach, a composite tool including items from our recommendations and other common criteria lists is tested in two ongoing systematic electronic reviews of prognosis. For each review, two sets of raters evaluate quality in a minimum of 30 primary studies. Items for each QA tool are extracted and graded according to the developers. Analyses include sensibility, validity, inter-rater reliability within tools and agreement between tools, and agreement with a judgment of overall study quality.
Results: We constructed a list of 378 distinct quality items and grouped them into 24 domains/ideas. Fourteen domains mapped onto a theoretical framework addressing potential opportunities for bias including: 1. study participation, 2. study attrition, 3. prognostic factor measurement, 4. confounding measurement and account, 5. outcome measurement, and 6. analysis. Adequate assessment of each potential biases ranged from 12.7% (for confounding measurement and account) to 77.5% (for prognostic factor measurement). Only two reviews assessed all parts of the theoretical model. We make recommendations to improve the quality assessment of future systematic reviews. Clinimetric testing results will be presented at the Cochrane Colloquium.
Conclusions: Systematic reviews are frequently used to summarize information on health care topics and guide policy. In many reviews of prognosis quality appraisal is inadequate and done with little standardization, calling into question the scientific validity of the results. We recommend quality appraisal focusing on key potential biases in reviews of prognosis. A transparent approach will help to improve this literature and facilitate future study on the importance of potential biases in the study of prognosis.