A comparison of long and short quality scores of published trials

Article type
Fardy JM, Macintosh DG
Various approaches have been used for the formal assessment of the methodological quality of trial reports. The 100-point protocol developed by Chalmers et al. is most widely known but time consuming to apply. A simpler 9-point protocol has also been used by the same group. The purpose of this study was to compare the two protocols in the assessment of a group of randomized trials. In a previous meta-analysis, 14 fully published trials of prophylactic sclerotherapy had been scored by two independent assessors, using masked copies of the publications and a modification of the 100-point protocol. For the purposes of this current study, the 14 trials were then re-scored by two different assessors using unmasked copies of the publications and the simpler 9 point protocol. The quality scores generated by the two protocols were compared using Spearman's rank correlation. The rho value (corrected for ties) was 0.924 (p = 0.0009). In the original meta-analysis, a sensitivity analysis based on quality score was carried out using trials with scores > 50 on the 100-point protocol. Using the 9-point scoring system and a cutoff of > 5, an identical list of superior quality trials was generated. This study suggests that a simple 9-point protocol can be substituted for the more detailed 100-point protocol in the assessment of published trials.