Article type
Year
Abstract
Background: Assessment of risk of bias (RoB) using information from publications of randomised controlled trials (RCTs) is common practice for systematic reviews. It has been proposed as a basis for adjusting the results of meta-analyses to account for trials assessed at high risk.
Objectives: We aimed to assess whether RoB assessments from trial reports are comparable to those using supplementary information obtained from trialists.
Methods: We sampled published RCTs included in our meta-analyses of individual participant data (IPDMA) for which trial protocols or forms detailing trial design were available. RoB was assessed using criteria outlined in the Cochrane Handbook and the Cochrane Risk of Bias tool (RevMan 5.1). Assessments were conducted for each trial based on trial reports only, or trial reports plus supplementary information from trialists, and were compared for consistency within domains and overall.
Results: Preliminary results are based on 25 RCTs from 3 IPDMAs in cancer. Using trial reports, the RoB was judged to be low (14 trials) or unclear (11 trials) for sequence generation and low (10 trials) or unclear (15 trials) for allocation concealment. However, supplementary information resulted in low RoB assessments for all trials for sequence generation and for 24 trials for allocation concealment. Selective reporting bias was judged to be high (14 trials) or unclear (10 trials) from publications alone, however supplementary data led to 24 trials being assessed as low RoB.
Conclusions: Supplementary information from trialists can alter high or unclear judgements of bias, particularly in regard to selective reporting of outcomes. Our results suggest review authors should be cautious in adjusting the results of meta-analyses based on RoB assessments from publications, particularly where assessments are unclear. Results will be updated to include 7 additional IPDMAs (76 RCTs) and the randomisation integrity will be further explored using IPD.
Objectives: We aimed to assess whether RoB assessments from trial reports are comparable to those using supplementary information obtained from trialists.
Methods: We sampled published RCTs included in our meta-analyses of individual participant data (IPDMA) for which trial protocols or forms detailing trial design were available. RoB was assessed using criteria outlined in the Cochrane Handbook and the Cochrane Risk of Bias tool (RevMan 5.1). Assessments were conducted for each trial based on trial reports only, or trial reports plus supplementary information from trialists, and were compared for consistency within domains and overall.
Results: Preliminary results are based on 25 RCTs from 3 IPDMAs in cancer. Using trial reports, the RoB was judged to be low (14 trials) or unclear (11 trials) for sequence generation and low (10 trials) or unclear (15 trials) for allocation concealment. However, supplementary information resulted in low RoB assessments for all trials for sequence generation and for 24 trials for allocation concealment. Selective reporting bias was judged to be high (14 trials) or unclear (10 trials) from publications alone, however supplementary data led to 24 trials being assessed as low RoB.
Conclusions: Supplementary information from trialists can alter high or unclear judgements of bias, particularly in regard to selective reporting of outcomes. Our results suggest review authors should be cautious in adjusting the results of meta-analyses based on RoB assessments from publications, particularly where assessments are unclear. Results will be updated to include 7 additional IPDMAs (76 RCTs) and the randomisation integrity will be further explored using IPD.