Article type
Year
Abstract
Background:
According to the GRADE system, guidelines panels consider four factors when making recommendations: quality of evidence, balance between the intervention’s benefits and harms, resource use and patient’s values and preferences. However, no study to date has assessed the association of these factors with the strength of recommendations.
Objective:
Empirical evaluation of the impact of these factors on the strength of recommendations in the setting of development of the guidelines for the use of platelets by the AABB (formerly, the American Association of Blood Banks).
Methods:
AABB convened a 17 member, multidisciplinary panel to develop guidelines for the use of platelets for 11 different clinical indications. One week after the meeting, panel members used a web-based survey tool to document their judgments of the strength of recommendation and all four GRADE factors. We used both a multilevel mixed effect logistic regression model and a simpler regression model to assess the relationship between the strength of recommendation and the GRADE factors. The results were similar, and we here present findings from the simpler model. We used Cronbach alpha to assess the consistency in rating of GRADE factors.
Results:
Cronbach alpha ranged from poor for judgment of the balance of benefits and harms (0.55) to good for judgment on the patients’ values and preferences (0.73). The quality of evidence was the only statistically significant (OR=4.5; p<0.001) GRADE factor associated with the strength of recommendations: the higher the quality of evidence, the higher the probability of a strong recommendation. The predictive model (Figure) suggested an approximately 90% probability of a strong recommendation when confidence in the effects of intervention was high, and 10% when the quality of evidence was very low.
Conclusion:
In this first comprehensive empirical study, we show that a guideline panel's confidence in estimates of effect is the major determinant of making a strong versus weak recommendation. This confirms the fundamental EBM view that when the quality of evidence is high, evidence indeed can serve as a neutral arbiter among competing views.
According to the GRADE system, guidelines panels consider four factors when making recommendations: quality of evidence, balance between the intervention’s benefits and harms, resource use and patient’s values and preferences. However, no study to date has assessed the association of these factors with the strength of recommendations.
Objective:
Empirical evaluation of the impact of these factors on the strength of recommendations in the setting of development of the guidelines for the use of platelets by the AABB (formerly, the American Association of Blood Banks).
Methods:
AABB convened a 17 member, multidisciplinary panel to develop guidelines for the use of platelets for 11 different clinical indications. One week after the meeting, panel members used a web-based survey tool to document their judgments of the strength of recommendation and all four GRADE factors. We used both a multilevel mixed effect logistic regression model and a simpler regression model to assess the relationship between the strength of recommendation and the GRADE factors. The results were similar, and we here present findings from the simpler model. We used Cronbach alpha to assess the consistency in rating of GRADE factors.
Results:
Cronbach alpha ranged from poor for judgment of the balance of benefits and harms (0.55) to good for judgment on the patients’ values and preferences (0.73). The quality of evidence was the only statistically significant (OR=4.5; p<0.001) GRADE factor associated with the strength of recommendations: the higher the quality of evidence, the higher the probability of a strong recommendation. The predictive model (Figure) suggested an approximately 90% probability of a strong recommendation when confidence in the effects of intervention was high, and 10% when the quality of evidence was very low.
Conclusion:
In this first comprehensive empirical study, we show that a guideline panel's confidence in estimates of effect is the major determinant of making a strong versus weak recommendation. This confirms the fundamental EBM view that when the quality of evidence is high, evidence indeed can serve as a neutral arbiter among competing views.