Article type
Year
Abstract
Background:
An outcome may be selected as a primary or a secondary outcome in a randomized clinical trial (RCT). Researchers may expect a larger effect size from this outcome when it is selected as a primary outcome than when it is selected as a secondary outcome, leading to expectation bias.
Objectives:
To compare the effect estimates between RCTs selecting an outcome as a primary outcome (RCT-POs) and those selecting the same outcome as a secondary outcome (RCT-SOs) assessing the same clinical question.
Methods:
This was a retrospective cohort study on RCTs. We screened six high-impact journals (the Annals of Internal Medicine, The BMJ, JAMA, JAMA Internal Medicine, The Lancet, and PLoS Medicine) published between 2017 and 2022 for meta-analyses that assessed the efficacy of clinical interventions and produced statistically significant estimates. In each meta-analysis, RCT-POs were included in the exposure group, whereas RCTs-SOs were in the control group. The ratio of risk ratio (RRR), hazard ratio (RHR), or odds ratio (ROR, with OR transformed from SMD) between RCTs-PO and RCTs-SO were pooled to form a single estimate by random-effects meta-analyses. We planned to include 100 meta-analyses.
Results:
This was an interim analysis. A total of 59 meta-analyses were identified, including 27 assessing medical products and 32 assessing other interventions. Six hundred six RCTs were included, comprising 259 RCT-POs and 347 RCT-SOs. On average, RCT-POs produced an effect estimate 1.18 (95% CI: 1.08-1.30, I2=44.0%) times greater than RCT-SOs. RCT-POs assessing medical products produced an effect estimate 1.24 (95% CI: 1.12-1.38; I2=44.3%) times greater than RCT-SOs, whereas RCT-POs assessing other interventions produced an effect estimate 1.09 (95%: 0.93-1.29; I2=44.7%) times greater than RCT-SOs.
Conclusions:
RCT-POs generally produced an effect estimate 18% greater than RCT-SOs, implying that the expectations bias could significantly distort the results of RCTs, especially those assessing the efficacy of medical products.
Patient, public, and/or healthcare consumer involvement:
An outcome may be selected as a primary or a secondary outcome in a randomized clinical trial (RCT). Researchers may expect a larger effect size from this outcome when it is selected as a primary outcome than when it is selected as a secondary outcome, leading to expectation bias.
Objectives:
To compare the effect estimates between RCTs selecting an outcome as a primary outcome (RCT-POs) and those selecting the same outcome as a secondary outcome (RCT-SOs) assessing the same clinical question.
Methods:
This was a retrospective cohort study on RCTs. We screened six high-impact journals (the Annals of Internal Medicine, The BMJ, JAMA, JAMA Internal Medicine, The Lancet, and PLoS Medicine) published between 2017 and 2022 for meta-analyses that assessed the efficacy of clinical interventions and produced statistically significant estimates. In each meta-analysis, RCT-POs were included in the exposure group, whereas RCTs-SOs were in the control group. The ratio of risk ratio (RRR), hazard ratio (RHR), or odds ratio (ROR, with OR transformed from SMD) between RCTs-PO and RCTs-SO were pooled to form a single estimate by random-effects meta-analyses. We planned to include 100 meta-analyses.
Results:
This was an interim analysis. A total of 59 meta-analyses were identified, including 27 assessing medical products and 32 assessing other interventions. Six hundred six RCTs were included, comprising 259 RCT-POs and 347 RCT-SOs. On average, RCT-POs produced an effect estimate 1.18 (95% CI: 1.08-1.30, I2=44.0%) times greater than RCT-SOs. RCT-POs assessing medical products produced an effect estimate 1.24 (95% CI: 1.12-1.38; I2=44.3%) times greater than RCT-SOs, whereas RCT-POs assessing other interventions produced an effect estimate 1.09 (95%: 0.93-1.29; I2=44.7%) times greater than RCT-SOs.
Conclusions:
RCT-POs generally produced an effect estimate 18% greater than RCT-SOs, implying that the expectations bias could significantly distort the results of RCTs, especially those assessing the efficacy of medical products.
Patient, public, and/or healthcare consumer involvement: