Incorporation of non-randomised evidence in systematic reviews of surgical interventions: a case study of endovascular versus open repair of abdominal aortic aneurysms

Article type
Authors
Fayter D, Chambers D, Paton F, Wright K, Woolacott N
Abstract
Background: Surgical procedures are more likely than drug interventions to be evaluated in non-randomised and observational studies. Systematic reviews of surgical interventions that are restricted to randomised controlled trials (RCTs) may therefore omit relevant information and lack clinical credibility. Objectives: Our objective was to note the limitations of the RCT evidence in a systematic review of endovascular aneurysm repair (EVAR) compared to open repair or non-surgical management and to assess the extent to which these limitations could be addressed by other sources of data. Methods: We conducted a review of EVAR versus open repair or non-surgical management of abdominal aortic aneurysms. Preliminary scoping work revealed the limitations of the RCT evidence, hence we included pre-specified registries and statistical risk modelling studies. Results: The evolution of EVAR devices meant that the available RCT data could be perceived as out of date. In addition, the trial inclusion criteria might not reflect clinical practice. Data from the most recent registry reports were synthesised to provide ‘real world’ estimates of key efficacy and safety outcomes. Furthermore, the trials dichotomised patients as fit/unfit for open surgery, which may not reflect the full spectrum of risk. Data from over 30 risk modelling studies were available that evaluated levels of baseline risks and their impact on outcome. Whilst the RCT data suggested no difference in overall all cause mortality between treatment groups, the results of the risk model studies could be used to inform further analyses of the RCT data or the design of further trials to identify patients likely to benefit most from EVAR. Conclusions and implications: Specific challenges in systematic reviews of surgical interventions demand adaptations to standard methods. Although there is a need to use the best evidence available, relying solely on RCTs may not be appropriate. Recognising the limitations of RCTs and using other evidence to resolve these should help to ensure that a review’s conclusions are timely, useful and generalisable.