Article type
Year
Abstract
Background: Conclusions in abstracts of randomised controlled trials (RCTs) should accurately reflect study results, as people may only read this section of the report. Authors sometimes use strategies (‘spin’) to suggest therapies are beneficial, even if results suggest otherwise. We examined how often, when abstracts of RCTs of interventions for asthma suggest an intervention has benefit, this claim is justifiable. To suggest interventional benefit, we felt studies must recruit the pre-specified sample size, and demonstrate statistically significant between-group differences for the primary outcome (p < 0.05).
Methods: Studies were identified from Cochrane Reviews of chronic asthma therapies. We included RCTs, of pharmacological and non-pharmacological/complementary interventions, in which the abstract conclusion suggested interventional benefit. The abstract conclusion of each RCT was categorised: (1) suggesting benefit without uncertainty; (2) suggesting benefit with uncertainty (eg more studies required); (3) not suggesting benefit. We identified whether the trial report presented a required sample size, primary outcome, and between-group comparison for this. For abstracts suggesting benefit without uncertainty, we evaluated whether the claim was (1) justified (sufficient participants, and primary outcome between-group comparison p < 0.05); (2) unjustified (not all criteria met); (3) non-evaluable (insufficient information). As an exploratory evaluation of ‘spin’, we evaluated how often, when abstract conclusions suggest benefit with uncertainty, this claim appeared based on secondary outcome/within-group comparisons.
Results: Of 153 abstracts screened, 42 full RCT reports were evaluated. 26/42 abstracts suggested benefit with no uncertainty. This was justified in 11/26, unjustified in 3/26, and non-evaluable in 12/26. 16/42 abstracts suggested benefit with uncertainty. In 11/16, the suggested benefit was felt to be based on secondary outcome/within-group comparisons (ie ‘spin’). Non-evaluable claims of benefit, and ‘spin’, were particularly common in RCTs of non-pharmacological/complementary therapies.
Conclusion: When abstracts of RCT reports suggest benefit of experimental treatment, the full report frequently includes insufficient information to justify this claim. Spin is commonplace in RCT abstracts.
Methods: Studies were identified from Cochrane Reviews of chronic asthma therapies. We included RCTs, of pharmacological and non-pharmacological/complementary interventions, in which the abstract conclusion suggested interventional benefit. The abstract conclusion of each RCT was categorised: (1) suggesting benefit without uncertainty; (2) suggesting benefit with uncertainty (eg more studies required); (3) not suggesting benefit. We identified whether the trial report presented a required sample size, primary outcome, and between-group comparison for this. For abstracts suggesting benefit without uncertainty, we evaluated whether the claim was (1) justified (sufficient participants, and primary outcome between-group comparison p < 0.05); (2) unjustified (not all criteria met); (3) non-evaluable (insufficient information). As an exploratory evaluation of ‘spin’, we evaluated how often, when abstract conclusions suggest benefit with uncertainty, this claim appeared based on secondary outcome/within-group comparisons.
Results: Of 153 abstracts screened, 42 full RCT reports were evaluated. 26/42 abstracts suggested benefit with no uncertainty. This was justified in 11/26, unjustified in 3/26, and non-evaluable in 12/26. 16/42 abstracts suggested benefit with uncertainty. In 11/16, the suggested benefit was felt to be based on secondary outcome/within-group comparisons (ie ‘spin’). Non-evaluable claims of benefit, and ‘spin’, were particularly common in RCTs of non-pharmacological/complementary therapies.
Conclusion: When abstracts of RCT reports suggest benefit of experimental treatment, the full report frequently includes insufficient information to justify this claim. Spin is commonplace in RCT abstracts.