Article type
Year
Abstract
Background: Differences in ranking, as a result of using different criteria lists, may cause problems when quality scores are incorporated into a systematic review or meta-analysis. Therefore, empirical evidence is needed to establish whether methods in quality assessment are valid and reliable
Objectives: To determine whether different type of quality assessment scales determine similar RCT quality ranking and affect the conclusion of meta-analytic studies.
Methods: A data set of 38 RCT concerning the efficacy of Alpha interferon therapy in chronic hepatitis B was used. The quality lists were the Maastricht list(M), the Jadad list(J), the Delphi list(D) and the suggested Cochrane Collaboration (Gold Standard) approach. Spearman Correlation Coefficient, Kappa agreement and weighted Kappa(wK) were used to compare the instruments and ranking. The main outcome were HBV-DNA and HbeAg clearence. The meta-analysis was done using different quality threshold.
Results: Most of studies showed methodological flaws. The Kappa agreement between reviewers depends on the method, set on fair to light (D=0.12; J=0,29; M=0,33). The methods correlation were moderate (D/J=0,51; D/M=0,53; J/M=0,52). When all studies were pooled in meta-analysis, against the best of then, the results were, respectively (HBV-DNA): RR=0,71; CI95% 0,66-0,77 and RR=0,67 CI95%: 0,58-0,79. With the Cochrane approach: RR=0,71; CI95%: 0,65-0,78. For HbeAg they were RR=0,85; CI95%: 0,80-0,90 and RR=0,85 CI95%: 0,77-0,93. The ranking agreement was light: wK= 0,16 (p=0,02).
Conclusion: Considering the time consumed to apply and low agreement between the scales and that there is no consensus about how to assess the studies quality, the Cochrane approach is the easier and most reliable method to assess quality of RCT, and we found no significant difference between the methods in effect size of pooled analysis.
Referecences: 1. Verhagen AP, Vet HC, Bie RA, Kessels AG, Boers M, Bouter LM, Knipschild PG. The Delphi List: a criteria list for quality assessment of RCT for conducting systematic review developed by consensus. J Clin Epidemiol. 1998;51:1235-41.
Objectives: To determine whether different type of quality assessment scales determine similar RCT quality ranking and affect the conclusion of meta-analytic studies.
Methods: A data set of 38 RCT concerning the efficacy of Alpha interferon therapy in chronic hepatitis B was used. The quality lists were the Maastricht list(M), the Jadad list(J), the Delphi list(D) and the suggested Cochrane Collaboration (Gold Standard) approach. Spearman Correlation Coefficient, Kappa agreement and weighted Kappa(wK) were used to compare the instruments and ranking. The main outcome were HBV-DNA and HbeAg clearence. The meta-analysis was done using different quality threshold.
Results: Most of studies showed methodological flaws. The Kappa agreement between reviewers depends on the method, set on fair to light (D=0.12; J=0,29; M=0,33). The methods correlation were moderate (D/J=0,51; D/M=0,53; J/M=0,52). When all studies were pooled in meta-analysis, against the best of then, the results were, respectively (HBV-DNA): RR=0,71; CI95% 0,66-0,77 and RR=0,67 CI95%: 0,58-0,79. With the Cochrane approach: RR=0,71; CI95%: 0,65-0,78. For HbeAg they were RR=0,85; CI95%: 0,80-0,90 and RR=0,85 CI95%: 0,77-0,93. The ranking agreement was light: wK= 0,16 (p=0,02).
Conclusion: Considering the time consumed to apply and low agreement between the scales and that there is no consensus about how to assess the studies quality, the Cochrane approach is the easier and most reliable method to assess quality of RCT, and we found no significant difference between the methods in effect size of pooled analysis.
Referecences: 1. Verhagen AP, Vet HC, Bie RA, Kessels AG, Boers M, Bouter LM, Knipschild PG. The Delphi List: a criteria list for quality assessment of RCT for conducting systematic review developed by consensus. J Clin Epidemiol. 1998;51:1235-41.