Article type
Year
Abstract
Background: Network meta-analysis (NMA) is a powerful method that simultaneously synthesizes evidence from studies addressing the same clinical question comparing multiple interventions. The method allows inferences based on direct and indirect comparisons in a network. However, NMA results are reliable only when the prerequisite assumptions are met. Of interest, the consistency assumption requires that direct and indirect evidence in a network is in agreement. The design-by-treatment (DBT) interaction model is considered the best method to date; however, its statistical properties have not been well studied for complex networks.
Objectives: To assess the Type I error and Power of the inconsistency estimator from the DBT interaction model in triangular networks with arms denoted A, B, and C.
Methods: A simulation study in which, over 10,000 repeated iterations, we will simulate network meta-analysis data over a range of scenarios, fit frequentist, random-effect network meta-analysis models, and estimate the inconsistency of the network using the DBT model. From the 10,000 iterations, we estimate the estimator’s Power and Type I error based on the observed p-values. We consider varying values for the true odds ratio for the AB- (i.e., 0.65, 1.2, and 1.4) and AC-comparison (0.75, 1, and 1.4), inconsistency factor (0, 0.3, and 1), and number of studies between comparisons (1, 2, 5, and 10).
Preliminary Results: The power of the inconsistency test ranged from approximately 0.5 to 0.75, depending on the simulation scenario. Furthermore, the Type I error of the test ranged from approximately 0.4 to 0.45. Preliminary results indicate that the main driver of Power and Type I error is the assumed inconsistency factor in the data-generating mechanism.
Conclusions: Preliminary results indicate that the DBT inconsistency estimator suffers from a high Type I error and lacks sufficient Power to reliably detect inconsistency in a network. This suggests that further methodological work in assessing network inconsistency may be necessary so that NMAs used in informing decision-making are trustworthy. We intend to expand the simulations in our study to reflect other types of networks observed in practice (e.g., different geometries).
Patient, public and/or healthcare consumer involvement: Patients were not involved at this stage of the project.
Objectives: To assess the Type I error and Power of the inconsistency estimator from the DBT interaction model in triangular networks with arms denoted A, B, and C.
Methods: A simulation study in which, over 10,000 repeated iterations, we will simulate network meta-analysis data over a range of scenarios, fit frequentist, random-effect network meta-analysis models, and estimate the inconsistency of the network using the DBT model. From the 10,000 iterations, we estimate the estimator’s Power and Type I error based on the observed p-values. We consider varying values for the true odds ratio for the AB- (i.e., 0.65, 1.2, and 1.4) and AC-comparison (0.75, 1, and 1.4), inconsistency factor (0, 0.3, and 1), and number of studies between comparisons (1, 2, 5, and 10).
Preliminary Results: The power of the inconsistency test ranged from approximately 0.5 to 0.75, depending on the simulation scenario. Furthermore, the Type I error of the test ranged from approximately 0.4 to 0.45. Preliminary results indicate that the main driver of Power and Type I error is the assumed inconsistency factor in the data-generating mechanism.
Conclusions: Preliminary results indicate that the DBT inconsistency estimator suffers from a high Type I error and lacks sufficient Power to reliably detect inconsistency in a network. This suggests that further methodological work in assessing network inconsistency may be necessary so that NMAs used in informing decision-making are trustworthy. We intend to expand the simulations in our study to reflect other types of networks observed in practice (e.g., different geometries).
Patient, public and/or healthcare consumer involvement: Patients were not involved at this stage of the project.