Article type
Year
Abstract
Background:
Generalizability of evidence from randomized controlled trials (RCTs) is critical for clinical decisions and for the appraisal of current evidence in systematic reviews. The international classification of functioning (ICF) framework proposes a bio-psycho-social context when describing patients' conditions.
Objectives:
To assess the generalizability of evidence from RCTs published in the four leading medical journals, considering the comprehensiveness of patient description.
Methods:
All RCTs in the BMJ, JAMA, the Lancet and the New England Journal of Medicine (NEJM) from 1 January to 30 September 2017 were searched by hand and data were extracted. The papers were assessed using the CONSORT statement criteria and the criteria for observational effectiveness studies, the 'benchmarking controlled trials' (BCTs) framework.
Results:
Of the 161 RCTs, four were published in the BMJ, 50 in JAMA, 71 in the Lancet and 36 in the NEJM. Seventy-two RCTs were conducted in the US or Europe and 70 were multinational. Sixty-two per cent studied pharmacological interventions, 21% other conservative interventions, 7% surgical interventions, 3% rehabilitation interventions and the rest combinations of these. The patient's path before randomization was described in 13.7% of papers, reasons for exclusions were reported in 62.1% and healthcare settings were described in 7.5% of trials. Demographic and disorder-specific data were provided in 97.5% of papers, but any measure of functioning (disorder-specific or generic disability, or health-related quality of life) in 48.4%, at least two co-morbidities in 34.8%, any behavioral factor (alcohol consumption, smoking, physical activity or obesity) in 50.9%, any environmental factor (work and living conditions, marital status) in 10.6%, and any factor related to potential inequity (education, socioeconomic status, deprivation, ethnicity) in 50.3% of papers. There were major differences between journals in the completeness of reporting.
Conclusions:
Most of the RCTs were conducted in industrialized countries and on pharmacological or conservative therapy. Deficiencies in the description of patient selection, study setting and patient characteristics are common. These deficiencies hamper the generalizability of evidence for clinical decision-making and for systematic reviews. The use of the CONSORT statement and the BCT framework may widen the description of patients in RCTs and increase our ability to generalize evidence.
Generalizability of evidence from randomized controlled trials (RCTs) is critical for clinical decisions and for the appraisal of current evidence in systematic reviews. The international classification of functioning (ICF) framework proposes a bio-psycho-social context when describing patients' conditions.
Objectives:
To assess the generalizability of evidence from RCTs published in the four leading medical journals, considering the comprehensiveness of patient description.
Methods:
All RCTs in the BMJ, JAMA, the Lancet and the New England Journal of Medicine (NEJM) from 1 January to 30 September 2017 were searched by hand and data were extracted. The papers were assessed using the CONSORT statement criteria and the criteria for observational effectiveness studies, the 'benchmarking controlled trials' (BCTs) framework.
Results:
Of the 161 RCTs, four were published in the BMJ, 50 in JAMA, 71 in the Lancet and 36 in the NEJM. Seventy-two RCTs were conducted in the US or Europe and 70 were multinational. Sixty-two per cent studied pharmacological interventions, 21% other conservative interventions, 7% surgical interventions, 3% rehabilitation interventions and the rest combinations of these. The patient's path before randomization was described in 13.7% of papers, reasons for exclusions were reported in 62.1% and healthcare settings were described in 7.5% of trials. Demographic and disorder-specific data were provided in 97.5% of papers, but any measure of functioning (disorder-specific or generic disability, or health-related quality of life) in 48.4%, at least two co-morbidities in 34.8%, any behavioral factor (alcohol consumption, smoking, physical activity or obesity) in 50.9%, any environmental factor (work and living conditions, marital status) in 10.6%, and any factor related to potential inequity (education, socioeconomic status, deprivation, ethnicity) in 50.3% of papers. There were major differences between journals in the completeness of reporting.
Conclusions:
Most of the RCTs were conducted in industrialized countries and on pharmacological or conservative therapy. Deficiencies in the description of patient selection, study setting and patient characteristics are common. These deficiencies hamper the generalizability of evidence for clinical decision-making and for systematic reviews. The use of the CONSORT statement and the BCT framework may widen the description of patients in RCTs and increase our ability to generalize evidence.