Article type
Year
Abstract
Background: Meta-analyses of a baseline variable should have no heterogeneity as differences between treatment groups from true randomisation are due to chance. Heterogeneity of baseline variables has therefore been proposed to evaluate the robustness of randomization of randomised clinical trials (RCTs) in meta-analyses. Trials resulting in baseline heterogeneity should be excluded to reduce potential bias in meta-analyses. However, this method has not been applied widely and it is unclear whether this method is comparable to other objective tools to assess bias arising from randomisation such as Monte Carlo simulations. Recently, we assessed the randomisation in RCTs included in two Cochrane reviews by using Monte Carlo simulations and found that the randomisation of RCTs in one review was likely to be robust, while the other was not.
Objectives: To assess whether the two methods assessing randomisation, i.e. investigating the heterogeneity of baseline variables and Monte Carlo simulations, coincide in conclusions; and to assess the changes in risk estimates by excluding trials resulting in heterogeneity of baseline variables in meta-analyses.
Methods: We applied the two methods assessing randomisation to two Cochrane reviews evaluating endometrial scratching: one for in-vitro fertilization (IVF) and the other for intrauterine insemination (IUI)/natural intercourse. We extracted baseline age and body mass index (BMI) across treatment arms from trials with full-text publications included in the two Cochrane reviews. Next, we performed a fixed effects meta-analysis of the baseline age and BMI, respectively, and compared the results from baseline heterogeneity with those from Monte Carlo simulations. When baseline heterogeneity is observed, we excluded trials with the largest t-statistic and repeated the meta-analysis until I-square equals to 0.
Results: For RCTs included in the Cochrane review on endometrial scratching for IVF, there was no heterogeneity for baseline age or BMI, indicating that these RCTs were likely to be properly randomised, which is in line with the results of Monte Carlo simulations (p=0.8654). For trials included in the Cochrane review on endometrial scratching for IUI/intercourse, the heterogeneity for baseline age and BMI were high (I2=80% and 92%, respectively), indicating that some of these trials were unlikely to be properly randomised, which also agreed with the results of Monte Carlo simulations (p= 1.754*10-5). After excluding trials resulting in heterogeneity of baseline age or BMI, the effect size of meta-analysis changed from 2.02 (1.52, 2.68) to 1.76 (0.74, 4.21).
Conclusions: Assessing baseline heterogeneity could be an alternative method to evaluate bias arising from the randomisation process for trials included in a meta-analysis. Excluding trials contributing to baseline heterogeneity could result in a more accurate estimate of effect size in a meta-analysis.
Patient or healthcare consumer involvement: None.
Objectives: To assess whether the two methods assessing randomisation, i.e. investigating the heterogeneity of baseline variables and Monte Carlo simulations, coincide in conclusions; and to assess the changes in risk estimates by excluding trials resulting in heterogeneity of baseline variables in meta-analyses.
Methods: We applied the two methods assessing randomisation to two Cochrane reviews evaluating endometrial scratching: one for in-vitro fertilization (IVF) and the other for intrauterine insemination (IUI)/natural intercourse. We extracted baseline age and body mass index (BMI) across treatment arms from trials with full-text publications included in the two Cochrane reviews. Next, we performed a fixed effects meta-analysis of the baseline age and BMI, respectively, and compared the results from baseline heterogeneity with those from Monte Carlo simulations. When baseline heterogeneity is observed, we excluded trials with the largest t-statistic and repeated the meta-analysis until I-square equals to 0.
Results: For RCTs included in the Cochrane review on endometrial scratching for IVF, there was no heterogeneity for baseline age or BMI, indicating that these RCTs were likely to be properly randomised, which is in line with the results of Monte Carlo simulations (p=0.8654). For trials included in the Cochrane review on endometrial scratching for IUI/intercourse, the heterogeneity for baseline age and BMI were high (I2=80% and 92%, respectively), indicating that some of these trials were unlikely to be properly randomised, which also agreed with the results of Monte Carlo simulations (p= 1.754*10-5). After excluding trials resulting in heterogeneity of baseline age or BMI, the effect size of meta-analysis changed from 2.02 (1.52, 2.68) to 1.76 (0.74, 4.21).
Conclusions: Assessing baseline heterogeneity could be an alternative method to evaluate bias arising from the randomisation process for trials included in a meta-analysis. Excluding trials contributing to baseline heterogeneity could result in a more accurate estimate of effect size in a meta-analysis.
Patient or healthcare consumer involvement: None.