Article type
Year
Abstract
Background: randomized controlled trials (RCTs), including pragmatic clinical trials, provide the most rigorous level of evidence for comparative effectiveness, but most research studies remain observational due to a variety of factors (e.g. ethics, feasibility, and generalizability). The dominant methodological limitation inherent to observational studies is selection bias, which leads to confounding by indication or severity that then limits their use as evidence of comparative effectiveness. Specifically, individual studies control analytically for these confounders in a haphazard way, which makes pooling of data from observational studies challenging, due to the inclusion of different confounders across studies, and absence of outcome effect size data across these confounders.
Objectives: the objective of this study was to improve the diversity of methodologies that can be incorporated validly into comparative effectiveness evidence-based decision-making, by developing a statistical meta-analytic approach to control for confounding by indication or severity across multiple studies of similar and different methodologies.
Methods: this study used the case of preventive statin therapy, given its plethora of RCT and observational studies, to facilitate the proposed analysis. Following the standard systematic review approach recommended by the National Academy of Medicine to find, select, assess and synthesize effectiveness evidence, we included 90 relevant research studies in the analysis, from a total of 9962 articles identified. We extracted data for a variety of confounders, and performed meta-analyses for the RCTs and observational studies, which we pooled separately.
Results: to overcome the diverse confounding issues across studies, the authors developed a de-novo study-level index of confounding (or SLIC index), which consists of the raw difference in the value of a given confounder across the two intervention groups for each study. The approach taken included:
1) a priori definition of all variables qualified as confounders by indication or severity;
2) calculation of the SLIC index value for each confounder,
3) inclusion of each SLIC index as a separate variable in the meta-regressions; and
4) data imputation and performance of sensitivity analyses for analytic robustness.
The SLIC indices had consistent magnitudes across multiple meta-regressions.
Conclusions: this study demonstrates that it may possible to overcome two significant limitations of methodological diversity in comparative effectiveness studies by using the SLIC index to control for confounding by indication or severity. Additional research is needed to explore further the impact of data imputation methods for the SLIC index on the meta-regression in order to validate the utility of this statistical technique.
Patient or healthcare consumer involvement: lay healthcare consumers were involved in the conceptualization of the research question and its significance. They did not participate in the methodological aspects of the study.
Objectives: the objective of this study was to improve the diversity of methodologies that can be incorporated validly into comparative effectiveness evidence-based decision-making, by developing a statistical meta-analytic approach to control for confounding by indication or severity across multiple studies of similar and different methodologies.
Methods: this study used the case of preventive statin therapy, given its plethora of RCT and observational studies, to facilitate the proposed analysis. Following the standard systematic review approach recommended by the National Academy of Medicine to find, select, assess and synthesize effectiveness evidence, we included 90 relevant research studies in the analysis, from a total of 9962 articles identified. We extracted data for a variety of confounders, and performed meta-analyses for the RCTs and observational studies, which we pooled separately.
Results: to overcome the diverse confounding issues across studies, the authors developed a de-novo study-level index of confounding (or SLIC index), which consists of the raw difference in the value of a given confounder across the two intervention groups for each study. The approach taken included:
1) a priori definition of all variables qualified as confounders by indication or severity;
2) calculation of the SLIC index value for each confounder,
3) inclusion of each SLIC index as a separate variable in the meta-regressions; and
4) data imputation and performance of sensitivity analyses for analytic robustness.
The SLIC indices had consistent magnitudes across multiple meta-regressions.
Conclusions: this study demonstrates that it may possible to overcome two significant limitations of methodological diversity in comparative effectiveness studies by using the SLIC index to control for confounding by indication or severity. Additional research is needed to explore further the impact of data imputation methods for the SLIC index on the meta-regression in order to validate the utility of this statistical technique.
Patient or healthcare consumer involvement: lay healthcare consumers were involved in the conceptualization of the research question and its significance. They did not participate in the methodological aspects of the study.
PDF