Article type
Year
Abstract
Background: While randomized controlled trials (RCTs) are considered the gold standard for evidence on therapeutic interventions, they are susceptible to bias. There is a growing body of empirical evidence based on meta-epidemiological methods to quantify biases in RCTs; however, there are inconsistent findings across studies and clinical areas.
Objectives: To quantify bias related to specific methodological characteristics in child-relevant RCTs.
Methods: We identified systematic reviews containing a meta-analysis with 10–40 RCTs that were relevant to child health in the Cochrane Database of Systematic Reviews. Two reviewers independently assessed RCTs using items in the Cochrane Risk of Bias tool and other study factors. We used meta-epidemiological methods to assess for differences in effect estimates between studies classified as high/unclear vs. low risk of bias.
Results: We included 287 RCTs from 17 meta-analyses. The proportion of studies at high/unclear risk of bias was: 79% sequence generation, 83% allocation concealment, 67% blinding of participants, 47% blinding of outcome assessment, 49% incomplete outcome data, 32% selective outcome reporting, 44% other sources of bias, 97% overall risk of bias, 56% funding, 35% baseline imbalance, 13% blocked randomization in unblinded trials, and 1% early stopping for benefit. We found no significant differences in effect estimates for studies that were high/unclear vs. low risk of bias for any of the risk of bias domains, overall risk of bias, or other study factors.
Conclusions: We found no differences in effect estimates between studies based on risk of bias. A potential explanation is the small number of trials included. It has been postulated that much larger samples are needed to detect differences; however, this raises the question of the magnitude of differences if they exist. Until further evidence is available, reviewers should not exclude RCTs based solely on risk of bias particularly in the area of child health.
Objectives: To quantify bias related to specific methodological characteristics in child-relevant RCTs.
Methods: We identified systematic reviews containing a meta-analysis with 10–40 RCTs that were relevant to child health in the Cochrane Database of Systematic Reviews. Two reviewers independently assessed RCTs using items in the Cochrane Risk of Bias tool and other study factors. We used meta-epidemiological methods to assess for differences in effect estimates between studies classified as high/unclear vs. low risk of bias.
Results: We included 287 RCTs from 17 meta-analyses. The proportion of studies at high/unclear risk of bias was: 79% sequence generation, 83% allocation concealment, 67% blinding of participants, 47% blinding of outcome assessment, 49% incomplete outcome data, 32% selective outcome reporting, 44% other sources of bias, 97% overall risk of bias, 56% funding, 35% baseline imbalance, 13% blocked randomization in unblinded trials, and 1% early stopping for benefit. We found no significant differences in effect estimates for studies that were high/unclear vs. low risk of bias for any of the risk of bias domains, overall risk of bias, or other study factors.
Conclusions: We found no differences in effect estimates between studies based on risk of bias. A potential explanation is the small number of trials included. It has been postulated that much larger samples are needed to detect differences; however, this raises the question of the magnitude of differences if they exist. Until further evidence is available, reviewers should not exclude RCTs based solely on risk of bias particularly in the area of child health.