Article type
Year
Abstract
Background: Several meta-epidemiological studies have found an association between inadequate randomization and/or allocation concealment and an overestimation of treatment effect estimates (ES). No such studies have been conducted using continuous outcomes in allied health disciplines such as physical therapy (PT).
Objectives: To determine if adequacy of randomization and/or allocation concealment is associated with changes in ES when comparing PT trials with and without these methodological characteristics.
Methods: We identified randomized controlled trials (RCTs) included in meta-analyses in the PT discipline by searching the Cochrane Database of Systematic Reviews. Two reviewers assessed randomization sequence and allocation concealment independently. To determine the association between sequence generation, and allocation concealment and effect sizes, a two-level analysis was conducted using a meta-meta-analytic approach.
Results: A total of 393 trials included in 43 meta-analyses, with 44,622 patients contributed to this study. Adequate random sequence generation and appropriate allocation concealment were accomplished in only 39.7% and 11.5% of PT trials respectively. In addition, trials with inappropriate allocation concealment tended to overestimate effect sizes when compared with trials with adequate allocation concealment (198 RCTs: ES = 0.12; 95% CI -0.06 to 0.30). When pooling our results with another meta-epidemiological study that also investigated the effect of concealment of allocation using continuous outcomes, we obtained a pooled statistically significant value (ES = 0.14; 95% CI 0.02 to 0.26). There was no difference in ES in trials with appropriate or inappropriate random sequence generation (257 RCTs: ES = 0.02; 95% CI -0.12 to 0.15).
Conclusion: Our results suggest that when evaluating risk of bias of primary RCTs in the PT area, systematic reviewers and clinicians implementing research into practice should pay attention to these biases. Systematic reviewers should perform sensitivity analysis including trials with low risk of bias in these domains as primary analysis and/or in combination with less restrictive analyses.
Objectives: To determine if adequacy of randomization and/or allocation concealment is associated with changes in ES when comparing PT trials with and without these methodological characteristics.
Methods: We identified randomized controlled trials (RCTs) included in meta-analyses in the PT discipline by searching the Cochrane Database of Systematic Reviews. Two reviewers assessed randomization sequence and allocation concealment independently. To determine the association between sequence generation, and allocation concealment and effect sizes, a two-level analysis was conducted using a meta-meta-analytic approach.
Results: A total of 393 trials included in 43 meta-analyses, with 44,622 patients contributed to this study. Adequate random sequence generation and appropriate allocation concealment were accomplished in only 39.7% and 11.5% of PT trials respectively. In addition, trials with inappropriate allocation concealment tended to overestimate effect sizes when compared with trials with adequate allocation concealment (198 RCTs: ES = 0.12; 95% CI -0.06 to 0.30). When pooling our results with another meta-epidemiological study that also investigated the effect of concealment of allocation using continuous outcomes, we obtained a pooled statistically significant value (ES = 0.14; 95% CI 0.02 to 0.26). There was no difference in ES in trials with appropriate or inappropriate random sequence generation (257 RCTs: ES = 0.02; 95% CI -0.12 to 0.15).
Conclusion: Our results suggest that when evaluating risk of bias of primary RCTs in the PT area, systematic reviewers and clinicians implementing research into practice should pay attention to these biases. Systematic reviewers should perform sensitivity analysis including trials with low risk of bias in these domains as primary analysis and/or in combination with less restrictive analyses.