Article type
Year
Abstract
Background: the Cochrane 'Risk of bias' tool is frequently used for assessing the quality of Cochrane Reviews. So far no evaluation of 'other bias' (i.e. bias other than selection, performance, detection, attrition, or reporting) in Cochrane Reviews has been performed.
Objectives: to describe the risk of 'other bias' in Cochrane Reviews.
Methods: we conducted a survey of Cochrane Reviews of interventions, which were listed in the Cochrane Library in 2018. We abstracted data on the general characteristics of the reviews and the 'Other bias' in each randomized controlled trial (RCT) included in the reviews. Two review authors independently abstracted data from all eligible studies using a standardized form. We conducted descriptive data analyses including subgroup analyses by topic. We calculated the odds ratios for having low risk or unknown risk of other bias between the different subgroups.
Results: we identified a total of 611 eligible Cochrane Reviews involving 4882 RCTs. Of these, we identified 2187 (44.8%) as having low risk of other bias (e.g. not identified, not reported because the study was stopped early for benefit, none detected by the authors of the review, unlikely, no other known risks of bias, or baseline characteristics of the two groups being similar). We evaluated one-third of the RCTs (n = 1621; 33.2%) as having unclear risk (e.g. the trial was not registered, no detailed baseline characteristics, details of funding sources were not reported, no primary outcome defined, sample size calculation was not reported, study was supported by pharmaceutical company, or having no other known risks) and 649 (13.3%) RCTs as having high risk of other bias (e.g. lack of information for all risk of bias domains, no sample size calculations, insufficient information, contacting the corresponding author was unsuccessful, sponsored by pharmaceutical company, or low recruitment rate). The remaining 425 RCTs (8.7%) did not evaluate and report other bias. For low-risk evaluation, the topic of child health was higher than insurance medicine (odds ratio (OR) 3.47, P = 0.008) and neurology (OR = 4.19, P = 0.001). However, the unclear risk in child health fields was more common than the insurance medicine (OR = 2.46, P = 0.04) and insurance medicine (OR = 2.65, P = 0.009) in low-risk evaluation.
Conclusions: we judged only a few RCTs to have high risk of other bias; however in reality the risk may be greater. Cochrane Review authors should follow carefully the instructions in the Cochrane Handbook for Systematic Reviews of Interventions when conducting a Cochrane Review. In addition, the Cochrane Library should give more details and explanations of other bias in their Handbook to improve the quality and transparency of CRs.
Patient or healthcare consumer involvement: we hope this study could improve the reported rate of 'other bias' in risk of bias for the conduct of Cochrane Reviews.
Objectives: to describe the risk of 'other bias' in Cochrane Reviews.
Methods: we conducted a survey of Cochrane Reviews of interventions, which were listed in the Cochrane Library in 2018. We abstracted data on the general characteristics of the reviews and the 'Other bias' in each randomized controlled trial (RCT) included in the reviews. Two review authors independently abstracted data from all eligible studies using a standardized form. We conducted descriptive data analyses including subgroup analyses by topic. We calculated the odds ratios for having low risk or unknown risk of other bias between the different subgroups.
Results: we identified a total of 611 eligible Cochrane Reviews involving 4882 RCTs. Of these, we identified 2187 (44.8%) as having low risk of other bias (e.g. not identified, not reported because the study was stopped early for benefit, none detected by the authors of the review, unlikely, no other known risks of bias, or baseline characteristics of the two groups being similar). We evaluated one-third of the RCTs (n = 1621; 33.2%) as having unclear risk (e.g. the trial was not registered, no detailed baseline characteristics, details of funding sources were not reported, no primary outcome defined, sample size calculation was not reported, study was supported by pharmaceutical company, or having no other known risks) and 649 (13.3%) RCTs as having high risk of other bias (e.g. lack of information for all risk of bias domains, no sample size calculations, insufficient information, contacting the corresponding author was unsuccessful, sponsored by pharmaceutical company, or low recruitment rate). The remaining 425 RCTs (8.7%) did not evaluate and report other bias. For low-risk evaluation, the topic of child health was higher than insurance medicine (odds ratio (OR) 3.47, P = 0.008) and neurology (OR = 4.19, P = 0.001). However, the unclear risk in child health fields was more common than the insurance medicine (OR = 2.46, P = 0.04) and insurance medicine (OR = 2.65, P = 0.009) in low-risk evaluation.
Conclusions: we judged only a few RCTs to have high risk of other bias; however in reality the risk may be greater. Cochrane Review authors should follow carefully the instructions in the Cochrane Handbook for Systematic Reviews of Interventions when conducting a Cochrane Review. In addition, the Cochrane Library should give more details and explanations of other bias in their Handbook to improve the quality and transparency of CRs.
Patient or healthcare consumer involvement: we hope this study could improve the reported rate of 'other bias' in risk of bias for the conduct of Cochrane Reviews.