Article type
Year
Abstract
Background:
Public health interventions are often addressed as 'complex interventions'. Application of routine meta-analysis techniques to synthesise the results public health intervention studies provides a pooled estimate, which is diluted by the effect of complexity.
Objective:
To propose an alternative meta-analysis approach for complex public health interventions, which includes measuring the complexity and further adjusting the measured complexity in meta-analysis.
Methods:
We derived items in the tool from three different sources: theories deduced from a qualitative study, meticulous examination of published public health interventions and expert opinion. We assessed the reliability of the tool by computing intra-class correlation (ICC) for the scores from same set of 20 studies scored independently by three raters. We applied the tool to assess the complexity of 259 studies from 30 public health systematic reviews to determine the probability distribution of the complexity score. We considered the total complexity score as a covariate and adjusted this in meta-analysis through meta-regression. We carried out sensitivity analysis to determine the robustness of the estimate adjusted for complexity.
Results:
The tool consists of items to capture complexity in four domains, namely: population, intervention, context and outcome. We added the scores for all four domains to obtain a total complexity score. The ICC was 0.85 for the population domain, 0.74 for intervention, 0.43 for context, 0.81 for outcome and 0.69 for the total complexity score. We found lognormal distribution to be the best fit for the complexity score. From the meta-regression analysis, it was evident that the pooled estimate adjusted for complexity differed from the unadjusted pooled estimate. The adjusted estimate had a wider 95% confidence interval (CI) than the unadjusted estimate. Furthermore, the I2 statistic and Tau2 of the adjusted estimates were higher than those for unadjusted estimates. Sensitivity analysis showed that the estimates were not sensitive to the extreme scores.
Conclusion:
This new approach of measuring complexity in public health interventions and adjusting for these complexities with the help of a score in meta-analysis is a unique approach for public health evidence consolidation.
Patient or healthcare consumer involvement:
No.
Public health interventions are often addressed as 'complex interventions'. Application of routine meta-analysis techniques to synthesise the results public health intervention studies provides a pooled estimate, which is diluted by the effect of complexity.
Objective:
To propose an alternative meta-analysis approach for complex public health interventions, which includes measuring the complexity and further adjusting the measured complexity in meta-analysis.
Methods:
We derived items in the tool from three different sources: theories deduced from a qualitative study, meticulous examination of published public health interventions and expert opinion. We assessed the reliability of the tool by computing intra-class correlation (ICC) for the scores from same set of 20 studies scored independently by three raters. We applied the tool to assess the complexity of 259 studies from 30 public health systematic reviews to determine the probability distribution of the complexity score. We considered the total complexity score as a covariate and adjusted this in meta-analysis through meta-regression. We carried out sensitivity analysis to determine the robustness of the estimate adjusted for complexity.
Results:
The tool consists of items to capture complexity in four domains, namely: population, intervention, context and outcome. We added the scores for all four domains to obtain a total complexity score. The ICC was 0.85 for the population domain, 0.74 for intervention, 0.43 for context, 0.81 for outcome and 0.69 for the total complexity score. We found lognormal distribution to be the best fit for the complexity score. From the meta-regression analysis, it was evident that the pooled estimate adjusted for complexity differed from the unadjusted pooled estimate. The adjusted estimate had a wider 95% confidence interval (CI) than the unadjusted estimate. Furthermore, the I2 statistic and Tau2 of the adjusted estimates were higher than those for unadjusted estimates. Sensitivity analysis showed that the estimates were not sensitive to the extreme scores.
Conclusion:
This new approach of measuring complexity in public health interventions and adjusting for these complexities with the help of a score in meta-analysis is a unique approach for public health evidence consolidation.
Patient or healthcare consumer involvement:
No.