Various approaches have been used for the formal assessment of the methodological quality of trial reports. The 100-point protocol developed by Chalmers et al. is most widely known but time consuming to apply. A simpler 9-point protocol has also been used by the same group. The purpose of this study was to compare the two protocols in the assessment of a group of randomized trials. In a previous meta-analysis, 14 fully published trials of prophylactic sclerotherapy had been scored by two independent assessors, using masked copies of the publications and a modification of the 100-point protocol. For the purposes of this current study, the 14 trials were then re-scored by two different assessors using unmasked copies of the publications and the simpler 9 point protocol. The quality scores generated by the two protocols were compared using Spearman's rank correlation. The rho value (corrected for ties) was 0.924 (p = 0.0009). In the original meta-analysis, a sensitivity analysis based on quality score was carried out using trials with scores > 50 on the 100-point protocol. Using the 9-point scoring system and a cutoff of > 5, an identical list of superior quality trials was generated. This study suggests that a simple 9-point protocol can be substituted for the more detailed 100-point protocol in the assessment of published trials.
A comparison of long and short quality scores of published trials
More like this
- Assessing trial quality by a short and a detailed list
- Comparison of Medline and Embase retrieval of RCTs of the effects of educational interventions on asthma-related outcomes
- Interventions for osteoporosis: a comparison of retrieval rates for RCTs using a broad labour-intensive search strategy, and Cochrane searching algorithms
- Comparison of the tests of homogeneity based on odds ratio and risk difference for interaction
- Quality assessments of randomized controlled trials: an evaluation by the Chalmers versus the Jadad method