Article type
Year
Abstract
Background: ROBINS-I is the preferred tool to assess risk of bias in Cochrane Reviews for non-randomized studies (NRS). Whereas the GRADE Working Group typically assigns a low certainty to the body of evidence in these studies, the underlying study design is not considered as a risk of bias feature in ROBINS-I. Thus, the initial certainty of the body of evidence from NRS would be high allowing for a better comparison of evidence from RCTs and NRS. However, there is to date little if any methodological guidance on how to combine and assess the body of evidence originating from both RCTs and NRS in GRADE assessments in one systematic review.
Objectives: We aimed to combine results from RCTs and NRS for assessing the body of evidence in the framework of a systematic review evaluating the effectiveness of storage techniques for an avulsed tooth.
Methods: We searched for experimental and observational studies in the Cochrane Library, MEDLINE and Embase. Two reviewers independently assessed the risk of bias of the included studies using the Cochrane RoB 2 tool for RCTs and ROBINS-I for NRS. The certainty of the body of evidence was appraised according to the GRADE methodology.
Results: Out of 4118 references, 19 RCTs, 7 non-RCTs and 7 prospective cohort studies were included. The RCTs suffered from serious concerns regarding the randomization process, measurement of the outcome and selection of the reported results. The certainty of evidence was graded as very low after accounting for risk of bias (-1), indirectness (-1) and imprecision (-1).
In the cohort studies, there were also serious concerns regarding measurement of the outcome and selection of the reported results. Most of these studies controlled for confounding factors and there were only some concerns regarding selection or information bias. The certainty of evidence was downgraded for risk of bias (-1) and imprecision (-1). The cohort studies alleviated the concerns about indirectness. The cohort studies were hence rated as low certainty evidence and, counterintuitively, provided the best available evidence.
To tackle this discrepancy, we decided to assess the limitations in study design in cohort studies as very serious (-2). Thereby, we recognized that randomization is the only way to fully protect against confounding. No incompatibilities were noticed between the GRADE assessment of the RCTs and non-RCTs since the identified limitations were very similar.
Conclusions: ROBINS-I poses a number of challenges to summarize risk of bias in GRADE assessments when the results from RCTs and NRS are evaluated together. New guidance should take into account that the evaluation of RoB 2 and ROBINS-I tools cannot simply be brought together in GRADE evidence profiles and should be evaluated against the background of the RCT design.
Patient or healthcare consumer involvement: The interpretation of systematic review findings by experts and patients in formulating recommendations is only valid when they can rely on proper assessments of the body of evidence.
Objectives: We aimed to combine results from RCTs and NRS for assessing the body of evidence in the framework of a systematic review evaluating the effectiveness of storage techniques for an avulsed tooth.
Methods: We searched for experimental and observational studies in the Cochrane Library, MEDLINE and Embase. Two reviewers independently assessed the risk of bias of the included studies using the Cochrane RoB 2 tool for RCTs and ROBINS-I for NRS. The certainty of the body of evidence was appraised according to the GRADE methodology.
Results: Out of 4118 references, 19 RCTs, 7 non-RCTs and 7 prospective cohort studies were included. The RCTs suffered from serious concerns regarding the randomization process, measurement of the outcome and selection of the reported results. The certainty of evidence was graded as very low after accounting for risk of bias (-1), indirectness (-1) and imprecision (-1).
In the cohort studies, there were also serious concerns regarding measurement of the outcome and selection of the reported results. Most of these studies controlled for confounding factors and there were only some concerns regarding selection or information bias. The certainty of evidence was downgraded for risk of bias (-1) and imprecision (-1). The cohort studies alleviated the concerns about indirectness. The cohort studies were hence rated as low certainty evidence and, counterintuitively, provided the best available evidence.
To tackle this discrepancy, we decided to assess the limitations in study design in cohort studies as very serious (-2). Thereby, we recognized that randomization is the only way to fully protect against confounding. No incompatibilities were noticed between the GRADE assessment of the RCTs and non-RCTs since the identified limitations were very similar.
Conclusions: ROBINS-I poses a number of challenges to summarize risk of bias in GRADE assessments when the results from RCTs and NRS are evaluated together. New guidance should take into account that the evaluation of RoB 2 and ROBINS-I tools cannot simply be brought together in GRADE evidence profiles and should be evaluated against the background of the RCT design.
Patient or healthcare consumer involvement: The interpretation of systematic review findings by experts and patients in formulating recommendations is only valid when they can rely on proper assessments of the body of evidence.