Article type
Year
Abstract
Background: We conducted a systematic review to examine the possible association between the use of benzodiazepine derivatives and benzodiazepine related drugs and mortality. Non-randomised studies were included in the systematic review. We assessed risk of bias using the Cochrane Risk Of Bias Assessment Tool: for Non-Randomized Studies of Interventions (ACROBAT-NRSI).
Objectives: To present the agreement between reviewers who used ACROBAT-NRSI to assess the risk of bias of studies included in this systematic review.
Methods: Three reviewers (with experience in assessing risk of bias) independently examined the risk of bias of 33 prospective cohort studies. Inter-rater reliability of the bias assessment was estimated by calculating kappa statistics (k) using Stata. This was performed for each domain of bias separately and for the final overall assessment. Agreement was categorized as poor (K < 0.01), slight (k = 0.01 to 0.20), fair (K = 0.21 to 0.40), moderate (K = 0.41 to 0.60), substantial (K = 0.61 to 0.80), or almost perfect (0.81 to 1.00).
Results: Overall, assessed studies were judged as being at serious or critical risk of bias. There was slight agreement for the overall assessment of risk of bias (k = 0.03). For the seven domains of bias, agreement ranged from k = -0.19 to K = 0.35. There was poor agreement for two domains (bias in measurement of interventions and due to departures from intended interventions), slight for two (bias in selection of participants into the study and in selection of the reported result), and fair for three (bias due to confounding, due to missing data, in measurement of outcomes).
Conclusions: The inter-rater agreement varied across the domains of the risk of bias tool, ranging from poor to fair. While we had slight agreement for the overall assessment of risk of bias, all reviewers independently assessed overall risk of bias of examined studies to be either serious or critical.
Objectives: To present the agreement between reviewers who used ACROBAT-NRSI to assess the risk of bias of studies included in this systematic review.
Methods: Three reviewers (with experience in assessing risk of bias) independently examined the risk of bias of 33 prospective cohort studies. Inter-rater reliability of the bias assessment was estimated by calculating kappa statistics (k) using Stata. This was performed for each domain of bias separately and for the final overall assessment. Agreement was categorized as poor (K < 0.01), slight (k = 0.01 to 0.20), fair (K = 0.21 to 0.40), moderate (K = 0.41 to 0.60), substantial (K = 0.61 to 0.80), or almost perfect (0.81 to 1.00).
Results: Overall, assessed studies were judged as being at serious or critical risk of bias. There was slight agreement for the overall assessment of risk of bias (k = 0.03). For the seven domains of bias, agreement ranged from k = -0.19 to K = 0.35. There was poor agreement for two domains (bias in measurement of interventions and due to departures from intended interventions), slight for two (bias in selection of participants into the study and in selection of the reported result), and fair for three (bias due to confounding, due to missing data, in measurement of outcomes).
Conclusions: The inter-rater agreement varied across the domains of the risk of bias tool, ranging from poor to fair. While we had slight agreement for the overall assessment of risk of bias, all reviewers independently assessed overall risk of bias of examined studies to be either serious or critical.