A comparison of long and short quality scores of published trials

Tags: Poster
Fardy JM, Macintosh DG

Various approaches have been used for the formal assessment of the methodological quality of trial reports. The 100-point protocol developed by Chalmers et al. is most widely known but time consuming to apply. A simpler 9-point protocol has also been used by the same group. The purpose of this study was to compare the two protocols in the assessment of a group of randomized trials. In a previous meta-analysis, 14 fully published trials of prophylactic sclerotherapy had been scored by two independent assessors, using masked copies of the publications and a modification of the 100-point protocol. For the purposes of this current study, the 14 trials were then re-scored by two different assessors using unmasked copies of the publications and the simpler 9 point protocol. The quality scores generated by the two protocols were compared using Spearman's rank correlation. The rho value (corrected for ties) was 0.924 (p = 0.0009). In the original meta-analysis, a sensitivity analysis based on quality score was carried out using trials with scores > 50 on the 100-point protocol. Using the 9-point scoring system and a cutoff of > 5, an identical list of superior quality trials was generated. This study suggests that a simple 9-point protocol can be substituted for the more detailed 100-point protocol in the assessment of published trials.