Article type
Year
Abstract
Background: Advocates of 'real world evidence' argue that when adequately adjusting for bias, real world data might be as valid as those of randomized controlled trials (RCTs). Telecardiology allows a direct comparison, since both RCTs and large registries are available.
Objectives: In the field of telecardiology, we aimed to assess the validity of results derived from non-randomised studies (NRS). To this end, we comprehensively compared the mortality results of NRS with those of a meta-analysis including RCTs only.
Methods: In January 2018, we searched for NRS examining telemonitoring via implantable cardioverter-defibrillators (ICDs) or cardiac resynchronization therapies (CRTs) in people with heart failure. We considered the 10 largest NRS adjusting for confounders and published after 2007 (date of first RCT published) with a parallel control group and a minimal size of 100 participants per study arm. We compared the mortality results of the NRS with those of a systematic review with meta-analysis consisting of 17 RCTs including 10,130 participants (last search: August 2017).
Results: We identified six NRS (number of patients ranging from 312 to 185,778). Follow-up ranged from one to five years and is thus longer than those of the RCTs (maximum three years). All six NRS showed a significant beneficial association with all-cause-mortality for patients followed remotely (hazard ratios for death ranging from 0.19 to 0.67 (see Table 1)). Results were comparable for NRS adjusting for at least one disease variable and NRS without adjusting for disease variables. These significant results stand in sharp contrast to those of the meta-analysis of RCTs (odds ratio: 0.92 (95% confidence interval 0.81 to 1.05), P = 0.733).
Conclusions: The large discrepancy between the results of NRS and RCT suggests that residual confounding is a substantial problem of 'real world evidence'. In telecardiology, confounding by indication and self-selection are the most likely explanation. Therefore, even when adjusting for disease variables, bias in NRS is likely and their results should be interpreted with caution. Given their limited validity regarding efficacy, registries should be used for other purposes, especially when RCT evidence is already available.
Patient or healthcare consumer involvement: Patients were not involved in the presented analysis.
Objectives: In the field of telecardiology, we aimed to assess the validity of results derived from non-randomised studies (NRS). To this end, we comprehensively compared the mortality results of NRS with those of a meta-analysis including RCTs only.
Methods: In January 2018, we searched for NRS examining telemonitoring via implantable cardioverter-defibrillators (ICDs) or cardiac resynchronization therapies (CRTs) in people with heart failure. We considered the 10 largest NRS adjusting for confounders and published after 2007 (date of first RCT published) with a parallel control group and a minimal size of 100 participants per study arm. We compared the mortality results of the NRS with those of a systematic review with meta-analysis consisting of 17 RCTs including 10,130 participants (last search: August 2017).
Results: We identified six NRS (number of patients ranging from 312 to 185,778). Follow-up ranged from one to five years and is thus longer than those of the RCTs (maximum three years). All six NRS showed a significant beneficial association with all-cause-mortality for patients followed remotely (hazard ratios for death ranging from 0.19 to 0.67 (see Table 1)). Results were comparable for NRS adjusting for at least one disease variable and NRS without adjusting for disease variables. These significant results stand in sharp contrast to those of the meta-analysis of RCTs (odds ratio: 0.92 (95% confidence interval 0.81 to 1.05), P = 0.733).
Conclusions: The large discrepancy between the results of NRS and RCT suggests that residual confounding is a substantial problem of 'real world evidence'. In telecardiology, confounding by indication and self-selection are the most likely explanation. Therefore, even when adjusting for disease variables, bias in NRS is likely and their results should be interpreted with caution. Given their limited validity regarding efficacy, registries should be used for other purposes, especially when RCT evidence is already available.
Patient or healthcare consumer involvement: Patients were not involved in the presented analysis.