Article type
Year
Abstract
Background: In meta-analyses, funnel plot asymmetry can be considered evidence of small study effects (possible publication bias (PB)). Egger’s test is a linear regression of the treatment effect estimates on their standard errors weighted by their inverse variance and is conducted as a formal statistical test for funnel plot asymmetry. The performance of Egger’s and related tests has been widely studied for binary outcomes but not for continuous outcomes. Baseline risk (BR) is an interaction of treatment effect with the severity of a condition measured by the observed treatment effect in the control group. In postoperative pain meta-analyses it has been shown that, on average, studies with higher BR (i.e. higher morphine consumption in the control group) will have larger standard deviations. If the treatment effect estimates are also dependent on BR then this may cause correlation between the outcome measure and standard errors which could result in funnel plot asymmetry even in the absence of PB. To overcome this we propose a new test for funnel plot asymmetry based on meta-regression residuals. The new test is a two-stage process in which BR is included as a study-level covariate in a meta-regression (MR) model before a regression-based asymmetry test using the MR residuals as the outcome and inverse sample size as the exploratory variable is performed.
Objectives: To evaluate and compare the performance of Egger’s test and the test of MR residuals for identifying funnel plot asymmetry.
Methods: 1) Application of Egger’s test and the test of MR residuals to 9 meta-analyses of postoperative analgesics measuring 24-hour morphine consumption. 2) Simulation study to formally evaluate the test of MR residuals considering each combination of BR and PB being present or not.
Results: 1) Egger’s test and the test of MR residuals identified funnel plot asymmetry in 6 and 2 (of 9) meta-analyses respectively. 2) Based on 10,000 simulated meta-analyses the test of MR residuals had similar power to Egger’s test when no BR and PB were simulated (63% versus 63%) and reduced type I errors when BR and no PB were simulated (60% versus 6%). It also had modest power to detect funnel plot asymmetry in the presence of treatment effects interacting with BR (40%).
Conclusions: Continuous outcomes are commonly measured on an absolute (mean) difference scale and it is not uncommon for the magnitude of the intervention effect to be related to response in the control arm (i.e. baseline risk). When this is the case funnel plots can appear highly asymmetric, even when PB is not present, since correlations exist between outcome and both effect size and standard error. We have shown that Egger’s test is potentially misleading for continuous outcomes and a test which regresses the residuals from a MR model, including BR as a study-level covariate, has better statistical properties.
Patient or healthcare consumer involvement: There was no patient involvement in this research however implications from this research will be discussed with patient representatives.
Objectives: To evaluate and compare the performance of Egger’s test and the test of MR residuals for identifying funnel plot asymmetry.
Methods: 1) Application of Egger’s test and the test of MR residuals to 9 meta-analyses of postoperative analgesics measuring 24-hour morphine consumption. 2) Simulation study to formally evaluate the test of MR residuals considering each combination of BR and PB being present or not.
Results: 1) Egger’s test and the test of MR residuals identified funnel plot asymmetry in 6 and 2 (of 9) meta-analyses respectively. 2) Based on 10,000 simulated meta-analyses the test of MR residuals had similar power to Egger’s test when no BR and PB were simulated (63% versus 63%) and reduced type I errors when BR and no PB were simulated (60% versus 6%). It also had modest power to detect funnel plot asymmetry in the presence of treatment effects interacting with BR (40%).
Conclusions: Continuous outcomes are commonly measured on an absolute (mean) difference scale and it is not uncommon for the magnitude of the intervention effect to be related to response in the control arm (i.e. baseline risk). When this is the case funnel plots can appear highly asymmetric, even when PB is not present, since correlations exist between outcome and both effect size and standard error. We have shown that Egger’s test is potentially misleading for continuous outcomes and a test which regresses the residuals from a MR model, including BR as a study-level covariate, has better statistical properties.
Patient or healthcare consumer involvement: There was no patient involvement in this research however implications from this research will be discussed with patient representatives.