Article type
Year
Abstract
Introduction: Few will question the use of very large randomised clinical trials (RCTs) as the gold standard in estimates of intervention efficacy, and increasing evidence indicate that small RCTs with inadequate control of bias overestimate intervention efficacy.
Objectives: To examine the influence of quality connected with bias in RCTs on estimates of intervention efficacy using high-quality RCTs and very large RCTs (n> 1000) as the gold standard.
Methods: The analysis includes 14 meta-analyses, covering several therapeutic areas, and comprising at least one very large RCT, consisting of 27 very large RCTs and 163 smaller RCTs (n<1000). The quality was assessed by means of a validated 5-point scale (Jadad et al. 1996 focusing on adequate generation of the allocation sequence, double-blinding and description of follow-up) and by components (allocation concealment plus scale items). Multiple logistic regression models explored the associations between quality and the likelihood of an unwanted event.
Results: Compared to the very large RCTs, low-quality RCTs (<2 points, n=59) produced larger estimates of intervention efficacy (p=0.01) overstating the odds ratios by 48% (95% Cl 0.69-0.14). The results obtained in high-quality RCTs (>3 points, n=104) and the very large RCTs were not significantly different. RCTs with unclear generation of the allocation sequence overstated odds-ratios by 51% (95% Cl 0.71-0.16, p<0.01), and RCTs with inadequate or unclear allocation concealment overstated odds-ratio by 40% (95% Cl 0.63-0.03, p= 0.036) compared to RCTs describing adequate methods. RCTs without double-blinding overstated odds-ratios with 55% (95% Cl 0.71-0.31, p<0.001) compared to double-blind RCTs.
Discussion: Low-quality RCTs produce exaggerated estimates of intervention efficacy whereas high-quality RCTs seem to predict the results of very large RCTs. RCTs with unclear generation of the allocation sequence, allocation concealment and double blinding overestimate intervention efficacy. Supported by The Danish Medical Research Council and The 1991 Pharmacy Foundation, DK.
Objectives: To examine the influence of quality connected with bias in RCTs on estimates of intervention efficacy using high-quality RCTs and very large RCTs (n> 1000) as the gold standard.
Methods: The analysis includes 14 meta-analyses, covering several therapeutic areas, and comprising at least one very large RCT, consisting of 27 very large RCTs and 163 smaller RCTs (n<1000). The quality was assessed by means of a validated 5-point scale (Jadad et al. 1996 focusing on adequate generation of the allocation sequence, double-blinding and description of follow-up) and by components (allocation concealment plus scale items). Multiple logistic regression models explored the associations between quality and the likelihood of an unwanted event.
Results: Compared to the very large RCTs, low-quality RCTs (<2 points, n=59) produced larger estimates of intervention efficacy (p=0.01) overstating the odds ratios by 48% (95% Cl 0.69-0.14). The results obtained in high-quality RCTs (>3 points, n=104) and the very large RCTs were not significantly different. RCTs with unclear generation of the allocation sequence overstated odds-ratios by 51% (95% Cl 0.71-0.16, p<0.01), and RCTs with inadequate or unclear allocation concealment overstated odds-ratio by 40% (95% Cl 0.63-0.03, p= 0.036) compared to RCTs describing adequate methods. RCTs without double-blinding overstated odds-ratios with 55% (95% Cl 0.71-0.31, p<0.001) compared to double-blind RCTs.
Discussion: Low-quality RCTs produce exaggerated estimates of intervention efficacy whereas high-quality RCTs seem to predict the results of very large RCTs. RCTs with unclear generation of the allocation sequence, allocation concealment and double blinding overestimate intervention efficacy. Supported by The Danish Medical Research Council and The 1991 Pharmacy Foundation, DK.