Article type
Abstract
Background
Assessing the risk of bias (RoB) of included studies is a crucial step in evidence syntheses to inform interpretation and translation of findings. Standardized tools, such as the Cochrane RoB tools, are commonly used to assess RoB for randomized controlled trials (RCTs). Traditionally, these rely on published information, which is often incomplete or unclear. Availability of raw line-by-line data, as in individual participant data (IPD) meta-analyses (MA), enables more in-depth assessments, yet guidance on these is lacking.
Objectives
To share lessons learned from the modified application of Cochrane RoB-1 and RoB-2 tools in the conduct of 2 case studies of IPD-MA.
Methods
We assessed 48 RCTs using Cochrane RoB-1 and 34 RCTs using RoB-2 across 2 IPD-MA on cord management at preterm birth and childhood obesity prevention, respectively. We consulted relevant literature and an international interdisciplinary expert advisory group to modify and enhance assessments using IPD. On completion, we conducted a workshop among RoB assessors within our team to gain feedback and suggestions for improvement.
Results
We identified and applied several IPD-specific adaptions across domains for both RoB-1 and RoB-2 and describe these in a preliminary toolkit. Availability of IPD enabled additional checks (eg, scrutinization of allocation patterns to check the rigor of randomization) and reduction of RoB in some instances (eg, opportunity to conduct intention-to-treat analyses, even when inappropriate analysis methods were used). Selective outcome reporting was not an RoB issue since availability of IPD negates reliance on published reports. Uncertainties in assessment (eg, of allocation concealment or blinding) based on published information could be clarified directly with study representatives in IPD assessments more easily, as direct communication had already been established during data sharing. It was possible to conduct enhanced RoB assessments in parallel with other IPD-MA processes (eg, data cleaning and integrity checks).
Conclusions
IPD-MA offers additional opportunities to assess and reduce RoB that can be leveraged by applying our preliminary toolkit.
Relevance and Importance to Patients
This guidance may be used by evidence synthesists to increase the accuracy of their RoB assessments and increase confidence in the evidence base and practice recommendations generated by IPD-MA.
Assessing the risk of bias (RoB) of included studies is a crucial step in evidence syntheses to inform interpretation and translation of findings. Standardized tools, such as the Cochrane RoB tools, are commonly used to assess RoB for randomized controlled trials (RCTs). Traditionally, these rely on published information, which is often incomplete or unclear. Availability of raw line-by-line data, as in individual participant data (IPD) meta-analyses (MA), enables more in-depth assessments, yet guidance on these is lacking.
Objectives
To share lessons learned from the modified application of Cochrane RoB-1 and RoB-2 tools in the conduct of 2 case studies of IPD-MA.
Methods
We assessed 48 RCTs using Cochrane RoB-1 and 34 RCTs using RoB-2 across 2 IPD-MA on cord management at preterm birth and childhood obesity prevention, respectively. We consulted relevant literature and an international interdisciplinary expert advisory group to modify and enhance assessments using IPD. On completion, we conducted a workshop among RoB assessors within our team to gain feedback and suggestions for improvement.
Results
We identified and applied several IPD-specific adaptions across domains for both RoB-1 and RoB-2 and describe these in a preliminary toolkit. Availability of IPD enabled additional checks (eg, scrutinization of allocation patterns to check the rigor of randomization) and reduction of RoB in some instances (eg, opportunity to conduct intention-to-treat analyses, even when inappropriate analysis methods were used). Selective outcome reporting was not an RoB issue since availability of IPD negates reliance on published reports. Uncertainties in assessment (eg, of allocation concealment or blinding) based on published information could be clarified directly with study representatives in IPD assessments more easily, as direct communication had already been established during data sharing. It was possible to conduct enhanced RoB assessments in parallel with other IPD-MA processes (eg, data cleaning and integrity checks).
Conclusions
IPD-MA offers additional opportunities to assess and reduce RoB that can be leveraged by applying our preliminary toolkit.
Relevance and Importance to Patients
This guidance may be used by evidence synthesists to increase the accuracy of their RoB assessments and increase confidence in the evidence base and practice recommendations generated by IPD-MA.