Article type
Year
Abstract
Background: Quality scores are often seen as a convenient method to incorporate quality into a systematic review. They may be used to weight a meta-analysis, to investigate the effect of quality on test performance using a combined quality score in a meta-regression analysis, or as a summary indicator of quality to include in tables of study results. The main problem with quality scores is determining how to weight each item to provide an overall quality score. There is no objective way of doing this and different methods are likely to produce different scores.
We are not aware of any work which has looked at the effect of using different weightings for the same quality assessment tool to produce an overall quality score or that has been done in the area of diagnostic accuracy studies.
Objective: To use different methods of weighting items included in QUADAS[1] to produce five different summary quality scores, and to examine the effects of incorporating these quality scores into the analysis of a systematic review.
Methods: We developed six different schemes for weighting QUADAS to produce an overall study quality score: equal weighting; equal weighting accounting for not clear ; weighting according to item type (source of bias, source of variation or quality of reporting); weighting based on evidence of the effects of biases in diagnostic accuracy studies; weighting based on the co-efficients of the individual quality items included in a meta-regression analysis of the data for the review in which the tool is being used; and a subjective scoring determined by the authors. These different weighting schemes will be used to produce six summary quality scores for each study.
We have selected two different data sets from an existing systematic review of the diagnosis and follow-up of UTI in children under 5 to use for this project.
Results: The results of incorporating the quality assessment into the review using the following methods will be discussed: (a) stratifying studies into high and low quality studies using the quality score - relative diagnostic odds ratios will be calculated to compare the effects of high and low quality studies using each of the quality scores; (b) studies will be ranked according to quality score and we will investigate whether the ranking of each study is different according to the method used to weight the quality scores; (c) we will use each of the quality scores to weight the meta-analysis; and (d) the effects of quality on test performance will be investigated by extending the SROC model include quality score as a continuous variable.
References: 1. Whiting P, Rutjes A, Reitsma J, Bossuyt P, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003; 3:25.
We are not aware of any work which has looked at the effect of using different weightings for the same quality assessment tool to produce an overall quality score or that has been done in the area of diagnostic accuracy studies.
Objective: To use different methods of weighting items included in QUADAS[1] to produce five different summary quality scores, and to examine the effects of incorporating these quality scores into the analysis of a systematic review.
Methods: We developed six different schemes for weighting QUADAS to produce an overall study quality score: equal weighting; equal weighting accounting for not clear ; weighting according to item type (source of bias, source of variation or quality of reporting); weighting based on evidence of the effects of biases in diagnostic accuracy studies; weighting based on the co-efficients of the individual quality items included in a meta-regression analysis of the data for the review in which the tool is being used; and a subjective scoring determined by the authors. These different weighting schemes will be used to produce six summary quality scores for each study.
We have selected two different data sets from an existing systematic review of the diagnosis and follow-up of UTI in children under 5 to use for this project.
Results: The results of incorporating the quality assessment into the review using the following methods will be discussed: (a) stratifying studies into high and low quality studies using the quality score - relative diagnostic odds ratios will be calculated to compare the effects of high and low quality studies using each of the quality scores; (b) studies will be ranked according to quality score and we will investigate whether the ranking of each study is different according to the method used to weight the quality scores; (c) we will use each of the quality scores to weight the meta-analysis; and (d) the effects of quality on test performance will be investigated by extending the SROC model include quality score as a continuous variable.
References: 1. Whiting P, Rutjes A, Reitsma J, Bossuyt P, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003; 3:25.