Article type
Year
Abstract
Objectives: Randomised controlled trials (RCTs) are regarded as the best method for assessing the effectiveness of a health care intervention but they are not always possible. Health care decision makers need guidance when deciding whether or not to act on evidence from non-randomised study designs which may be biased. Previous reviews have concluded that non-randomised studies exaggerate benefit, but they may have failed to take account of differences in the populations, interventions and outcomes studied. We compared estimates of effectiveness from RCTs and non-randomised studies of mammographic screening (MS) and folic acid supplementation (FAS), for which the intervention, population and outcome studied were relatively homogeneous. We also investigated the association between effect size and study quality.
Methods: Eligible studies were identified from the Cochrane Library, Medline (1966-), EMBASE (1980-), from personal paper collections, by contacting experts in the respective fields, and from the reference lists of papers already identified. Three reviewers assessed the quality of each paper on four dimensions: reporting, generalisability, and susceptibility to bias or confounding. Meta-regression was used to estimate associations of study design, study quality and heterogeneity amongst studies on effect size
Results: Thirty four eligible papers were identified, 17 evaluating MS and 17 FAS. For MS and FAS respectively, 8 and 4 papers were RCTs, 5 and 6 were non-randomised trials or cohort studies, and 3 and 6 were matched or unmatched case control studies; 1 MS and 1 FAS study used some other design. Both cohort and case control studies had lower total quality scores than RCTs; cohort studies had significantly lower scores than case control studies. No effect of quality and heterogeneity amongst studies was found after taking account of study design. Pooled relative risk estimates from RCTs (MS=0.85; PAS=0.28) and cohort studies (MS=0.82; FAS=0.27) were very similar, but significantly different from pooled estimates for case control studies (MS: 0.51, i.e. greater benefit; FAS: 0.72, i.e. less benefit).
Discussion: This study found no evidence to support the view that non-randomised cohort studies exaggerate the benefit of interventions, but this finding may not generalise to other interventions. There are several reasons why case control studies should give different estimates to RCTs and cohort studies; however, the direction of the discrepancy was inconsistent and may be intervention specific.
Methods: Eligible studies were identified from the Cochrane Library, Medline (1966-), EMBASE (1980-), from personal paper collections, by contacting experts in the respective fields, and from the reference lists of papers already identified. Three reviewers assessed the quality of each paper on four dimensions: reporting, generalisability, and susceptibility to bias or confounding. Meta-regression was used to estimate associations of study design, study quality and heterogeneity amongst studies on effect size
Results: Thirty four eligible papers were identified, 17 evaluating MS and 17 FAS. For MS and FAS respectively, 8 and 4 papers were RCTs, 5 and 6 were non-randomised trials or cohort studies, and 3 and 6 were matched or unmatched case control studies; 1 MS and 1 FAS study used some other design. Both cohort and case control studies had lower total quality scores than RCTs; cohort studies had significantly lower scores than case control studies. No effect of quality and heterogeneity amongst studies was found after taking account of study design. Pooled relative risk estimates from RCTs (MS=0.85; PAS=0.28) and cohort studies (MS=0.82; FAS=0.27) were very similar, but significantly different from pooled estimates for case control studies (MS: 0.51, i.e. greater benefit; FAS: 0.72, i.e. less benefit).
Discussion: This study found no evidence to support the view that non-randomised cohort studies exaggerate the benefit of interventions, but this finding may not generalise to other interventions. There are several reasons why case control studies should give different estimates to RCTs and cohort studies; however, the direction of the discrepancy was inconsistent and may be intervention specific.