Article type
Year
Abstract
Background
Previous research has suggested that increased risk of bias may be associated with exaggerated effect size. However, this association is unclear in trials about treatments of chronic medical conditions. These conditions are associated with significant morbidity, mortality and burden. Knowing the status and direction of bias can affect the confidence of endusers in this body of evidence.
Objectives
To evaluate the risk of bias in randomized trials evaluating a drug or a device used to treat chronic medical conditions and to determine if this risk is associated with the magnitude of treatment effect.
Methods
We identified meta-analyses with at least 5 randomized controlled trials (RCTs) published between 2007 and 2015 in 10 high impact general medical journals. Meta-analyses had to evaluate a medication or device for chronic medical conditions. We used Cochrane Collaboration Risk of Bias tool to evaluate the included RCTs. Mixed-effects random intercept regressions were used to evaluate the association of bias judgments (high risk, unknown risk and low risk) and the effect size of individual RCT.
Results
We analyzed risk of bias of 930 RCTs [average of 13 RCTs (5-48) and 922 patients (10-20,536) per meta-analysis]. Only a small proportion of the RCTs received a clear judgment of high risk of bias (2-14% across domains) but a substantial proportion had unknown risk of bias judgment (notably, allocation concealment, 62%). Despite the large number of included RCTs in regression analysis, there was no statistically significant association between any of the 7 items of risk of bias and the effect size (ratios of odds ratios with 95% confidence intervals overlapping 1.0, Table 1).
Conclusions:
The reporting of randomized controlled trials continues to show a substantial amount of unknown or unclear ratings. Meta-analyses about treatments of chronic medical conditions depend on trials with a small proportion of high risk of bias ratings. The lack of association between ratings and the effect size suggests that the direction of bias remain unpredictable.
Patient or healthcare consumer involvement:
Not performed.
Previous research has suggested that increased risk of bias may be associated with exaggerated effect size. However, this association is unclear in trials about treatments of chronic medical conditions. These conditions are associated with significant morbidity, mortality and burden. Knowing the status and direction of bias can affect the confidence of endusers in this body of evidence.
Objectives
To evaluate the risk of bias in randomized trials evaluating a drug or a device used to treat chronic medical conditions and to determine if this risk is associated with the magnitude of treatment effect.
Methods
We identified meta-analyses with at least 5 randomized controlled trials (RCTs) published between 2007 and 2015 in 10 high impact general medical journals. Meta-analyses had to evaluate a medication or device for chronic medical conditions. We used Cochrane Collaboration Risk of Bias tool to evaluate the included RCTs. Mixed-effects random intercept regressions were used to evaluate the association of bias judgments (high risk, unknown risk and low risk) and the effect size of individual RCT.
Results
We analyzed risk of bias of 930 RCTs [average of 13 RCTs (5-48) and 922 patients (10-20,536) per meta-analysis]. Only a small proportion of the RCTs received a clear judgment of high risk of bias (2-14% across domains) but a substantial proportion had unknown risk of bias judgment (notably, allocation concealment, 62%). Despite the large number of included RCTs in regression analysis, there was no statistically significant association between any of the 7 items of risk of bias and the effect size (ratios of odds ratios with 95% confidence intervals overlapping 1.0, Table 1).
Conclusions:
The reporting of randomized controlled trials continues to show a substantial amount of unknown or unclear ratings. Meta-analyses about treatments of chronic medical conditions depend on trials with a small proportion of high risk of bias ratings. The lack of association between ratings and the effect size suggests that the direction of bias remain unpredictable.
Patient or healthcare consumer involvement:
Not performed.