Integrating randomised and non-randomised studies in systematic reviews and its implications for GRADE: rationale, perceptions, and proposed methods

Article type
Year
Authors
Cuello C1, Morgan R1, Schünemann H1
1McMaster University, Department of Clinical Epidemiology and Biostatistics, Canada
Abstract
Background: Randomised studies (RS) are considered the ideal individual source of research evidence. Non-randomised studies of interventions (NRS) are critical to many areas of evaluation, yet they are commonly disregarded or separated from RS, and considered less certain due to confounding and bias. Using new tools for the assessment of NRS included in systematic reviews (eg, ROBINS-i) and GRADE (Grading of Recommendations, Assessment, Development and Evaluations) criteria, the integration of NRS with RS in systematic reviews could be more feasible.

Objectives: as part of a Cochrane Methods project, we set out to obtain the rationale, perceptions and methods used to integrate RS and NRS from a group of experts for integrating both bodies of evidence using GRADE.

Methods: We invited experts from different organizations (e.g. Cochrane, G-I-N (Guidelines International Network), GRADE members) to participate in a web-based survey to obtain their understanding, attitudes, and perceptions about integrating NRS with RS in a systematic review, and the integration within a summary of findings (SoF) table using GRADE. We assessed respondents’ preferences and rationale regarding the integration of RS and NRS on different possible GRADE scenarios based on certainty of the evidence.

Results: Of 187 initial responses, 137 (73.2%) were complete; 85% of respondents were highly experienced in systematic reviews and 65% had conducted at least one systematic review integrating RS and NRS. From presented scenarios, most experts favour a single SoF table differentiating RS from NRS (Fig). The situation most favourable for combining RS and NRS was when both bodies of evidence were of high certainty. A conceptual framework was drafted based on scenarios’ assessments, feedback, and individual responses.

Conclusions: Although most experts would prefer a single SoF table differentiating RS from NRS, we discuss other situations that are feasible for the RS/NRS integration based on the GRADE criteria. With more information and guidance on new methodological tools, the RS/NRS integration could help increase the certainty in the estimates in systematic reviews of interventions.