Article type
Abstract
Background: Cluster randomised trials (cRCTs) can lead to spurious conclusions if clustering is not taken into account during analysis. The inclusion of cRCTs with uncorrected unit of analysis errors in systematic reviews (SR) may lead to incorrect review conclusions.
Objectives: To determine the proportion of cRCTs that have unit of analysis errors (and whether they provide data to correct for errors) in a SR of diabetes quality improvement (QI) strategies.
Methods: Two researchers independently reviewed the 55 cRCTs to determine whether appropriate methods were used to adjust for clustering for the primary outcome: continuous HbA1c. If appropriate, we extracted the method of adjustment and the adjusted standard error (SE) (or reported data to calculate the adjusted SE), and the intraclass correlation coefficient (ICC). The total number of studies with persistent unit analysis errors requiring reviewer adjustment was determined.
Results: Of the 55 cRCTs, 37 (68%) accounted for clustering. Studies varied in the methods used to adjust for clustering (e.g. generalised estimating equations, mixed- effects regression) and over half (20/37) adjusted for additional covariates. Of the appropriately adjusted cRCTs, 2 studies reported SEs that could be directly extracted and 26 reported enough information (e.g., mean difference, p-values, confidence intervals, cluster number) from which an adjusted SE could be calculated. Eleven appropriately adjusted cRCTs did not report an adjusted SE or provide enough information from which one could be calculated. Ten studies (18%) reported ICCs for the HbA1c outcome, which ranged in value from -0.002 to 0.1. Combined with the 19 studies that did not account for clustering, 29 (52%) of studies required adjustment of their SEs with an internal (n=1) or externally-imputed (n=28) ICC.
Conclusions: Cluster RCTs pose important methodological challenges for SRs. Reviewers need to be aware of potential unit of analysis errors and adjust estimates accordingly. The presentation will outline our approach to identifying, and adjusting for, unit of analysis errors in our diabetes quality improvement SR.
Objectives: To determine the proportion of cRCTs that have unit of analysis errors (and whether they provide data to correct for errors) in a SR of diabetes quality improvement (QI) strategies.
Methods: Two researchers independently reviewed the 55 cRCTs to determine whether appropriate methods were used to adjust for clustering for the primary outcome: continuous HbA1c. If appropriate, we extracted the method of adjustment and the adjusted standard error (SE) (or reported data to calculate the adjusted SE), and the intraclass correlation coefficient (ICC). The total number of studies with persistent unit analysis errors requiring reviewer adjustment was determined.
Results: Of the 55 cRCTs, 37 (68%) accounted for clustering. Studies varied in the methods used to adjust for clustering (e.g. generalised estimating equations, mixed- effects regression) and over half (20/37) adjusted for additional covariates. Of the appropriately adjusted cRCTs, 2 studies reported SEs that could be directly extracted and 26 reported enough information (e.g., mean difference, p-values, confidence intervals, cluster number) from which an adjusted SE could be calculated. Eleven appropriately adjusted cRCTs did not report an adjusted SE or provide enough information from which one could be calculated. Ten studies (18%) reported ICCs for the HbA1c outcome, which ranged in value from -0.002 to 0.1. Combined with the 19 studies that did not account for clustering, 29 (52%) of studies required adjustment of their SEs with an internal (n=1) or externally-imputed (n=28) ICC.
Conclusions: Cluster RCTs pose important methodological challenges for SRs. Reviewers need to be aware of potential unit of analysis errors and adjust estimates accordingly. The presentation will outline our approach to identifying, and adjusting for, unit of analysis errors in our diabetes quality improvement SR.