Article type
Year
Abstract
Background: Risk of bias assessment is an integral part of a systematic review. The Cochrane risk of bias tool covers selection, performance, detection, attrition, and reporting biases. Trials are classed as ‘high’, ‘low’, or ‘unclear’ risk of bias. The ability to assess a trial confidently as ‘high’ or ‘low’ risk of bias is often hindered by poor reporting of methodology in publications.
Objectives: To explore whether access to trial protocols provides more definitive assessments of risk of bias than use of publications and conference abstracts.
Methods: As part of a systematic review and meta-analysis investigating the effectiveness of rhBMP-2 in spinal fusion we conducted risk of bias assessments for 12 published RCTs. We compared judgements about selection bias, performance bias, and detection bias for each trial based on information from trial publications and abstracts, from publicly available summary protocols and from detailed confidential trial protocols. Original detailed (in this case confidential) trial protocols were made available to us by the manufacturer of the product or through contacting study authors.
Results: Eleven RCTs were conducted by the manufacturer of rhBMP-2, and one was conducted independently. When confidential trial protocols were used, 45 of 48 risk of bias judgements were definitive (‘low’ or ‘high’). Risk of bias assessment based on publications and abstracts resulted in just 28 of 48 definitive judgements. This was due to non-reporting of randomisation and allocation concealment details in the publicly available sources. Publicly available protocols from trial registers produced no definitive judgments; the risk of bias was judged to be ‘unclear’ for all trials across all domains.
Conclusions: Reviewers can reduce uncertainty in risk of bias judgements by obtaining detailed trial protocols either from journals or directly from investigators or sponsors. Protocols exclusively from trial registers are unlikely to be adequate for this purpose.
Objectives: To explore whether access to trial protocols provides more definitive assessments of risk of bias than use of publications and conference abstracts.
Methods: As part of a systematic review and meta-analysis investigating the effectiveness of rhBMP-2 in spinal fusion we conducted risk of bias assessments for 12 published RCTs. We compared judgements about selection bias, performance bias, and detection bias for each trial based on information from trial publications and abstracts, from publicly available summary protocols and from detailed confidential trial protocols. Original detailed (in this case confidential) trial protocols were made available to us by the manufacturer of the product or through contacting study authors.
Results: Eleven RCTs were conducted by the manufacturer of rhBMP-2, and one was conducted independently. When confidential trial protocols were used, 45 of 48 risk of bias judgements were definitive (‘low’ or ‘high’). Risk of bias assessment based on publications and abstracts resulted in just 28 of 48 definitive judgements. This was due to non-reporting of randomisation and allocation concealment details in the publicly available sources. Publicly available protocols from trial registers produced no definitive judgments; the risk of bias was judged to be ‘unclear’ for all trials across all domains.
Conclusions: Reviewers can reduce uncertainty in risk of bias judgements by obtaining detailed trial protocols either from journals or directly from investigators or sponsors. Protocols exclusively from trial registers are unlikely to be adequate for this purpose.