Article type
Year
Abstract
Background: To date, few tools exist for assessing the risk of bias (RoB) from confounding variables in non-randomized studies (NRS).
Objectives: We tested a pilot Cochrane tool for RoB to assess its applicability and value in quantifying RoB in NRS.
Methods: We applied the Cochrane tool to a systematic review of NRS evaluating the effectiveness of lymph node dissection in patients undergoing radical cystectomy. Three independent review authors abstracted data on methodological quality using the Newcastle-Ottawa instrument and the Cochrane tool for assessing RoB.[1] The Cochrane tool assesses the precision, imbalance, and adjustment for each confounding variable, and overall RoB from confounding, blinding, incomplete outcome data and selective reporting, on 1 (low ROB) to 5 (high RoB) point scales. We pilot-tested the data abstraction form and developed detailed guidelines to optimize inter-observer agreement.
Results: Five NRS met the inclusion criteria. Confounders were identified in advance by literature search and responses to a web-based survey of eleven bladder cancer experts who were also asked to categorize what difference in the proportion of a confounding variable represented 'no’, a 'small’, 'moderate’ or 'large’ imbalance. To evaluate imbalance, each confounder was evaluated dichotomously (e.g. organ-confined versus non-organ-confined). Threshold values were derived using the median responses; e.g. for non-organ-confined disease, these were set at less than or equal to 2%, 2.1 to 5%, 5.1 to 7%, greater than 7%, respectively, thereby providing explicit anchors for imbalance criterion scoring.
Conclusions: The novel RoB tool represents a valuable instrument for quantifying the RoB of NRS that deserves further formal investigation in a larger, more diverse sample of studies. The described survey method appears an effective and efficient method to define the necessary imbalance criterion thresholds.
Reference
1. Reeves BC, Shea B, Wells GA. Classifying non-randomised studies (NRS) and the assessing the risk of bias for a systematic review. Workshop, 18th Cochrane Colloquium, Keystone 2010.
Objectives: We tested a pilot Cochrane tool for RoB to assess its applicability and value in quantifying RoB in NRS.
Methods: We applied the Cochrane tool to a systematic review of NRS evaluating the effectiveness of lymph node dissection in patients undergoing radical cystectomy. Three independent review authors abstracted data on methodological quality using the Newcastle-Ottawa instrument and the Cochrane tool for assessing RoB.[1] The Cochrane tool assesses the precision, imbalance, and adjustment for each confounding variable, and overall RoB from confounding, blinding, incomplete outcome data and selective reporting, on 1 (low ROB) to 5 (high RoB) point scales. We pilot-tested the data abstraction form and developed detailed guidelines to optimize inter-observer agreement.
Results: Five NRS met the inclusion criteria. Confounders were identified in advance by literature search and responses to a web-based survey of eleven bladder cancer experts who were also asked to categorize what difference in the proportion of a confounding variable represented 'no’, a 'small’, 'moderate’ or 'large’ imbalance. To evaluate imbalance, each confounder was evaluated dichotomously (e.g. organ-confined versus non-organ-confined). Threshold values were derived using the median responses; e.g. for non-organ-confined disease, these were set at less than or equal to 2%, 2.1 to 5%, 5.1 to 7%, greater than 7%, respectively, thereby providing explicit anchors for imbalance criterion scoring.
Conclusions: The novel RoB tool represents a valuable instrument for quantifying the RoB of NRS that deserves further formal investigation in a larger, more diverse sample of studies. The described survey method appears an effective and efficient method to define the necessary imbalance criterion thresholds.
Reference
1. Reeves BC, Shea B, Wells GA. Classifying non-randomised studies (NRS) and the assessing the risk of bias for a systematic review. Workshop, 18th Cochrane Colloquium, Keystone 2010.