A comparison of subgroup analyses in grant applications and publications

Article type
Authors
Boonacker C1, Hoes A1, van Liere-Visser K2, Schilder A3, Rovers M4
1Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Netherlands
2ZonMw, Netherlands Organisation for Health Research and Development, The Hague
3Department of Otorhinolaryn- gology, Wilhelmina Children’s Hospital, University Medical Center Utrecht, Netherlands & Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Netherlands
4Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Netherlands & Department of Otorhinolaryngology, Wilhelmina Children’s Hospital, University Medical Center Utrecht, Netherlands
Abstract
Background: One of the most important recommendations of the available guidelines on studying and reporting subgroup analyses, is to pre-specify subgroups rather than define them post hoc. We therefore studied both grant proposals and their publications and compared the subgroup analyses that were pre-specified in the grant proposal to those that were finally published.

Objectives: To compare grant applications and final publications regarding subgroup analyses.

Methods: Grants awarded by the 'Netherlands Organization for Health Research and Development’ from 2001, were studied. We analyzed whether or not projects mentioned subgroups in their grant application and related publications (i.e. the final report and scientific publications). The main outcome measure was the proportion of studies in which the publications were completely in agreement with the grant proposal, i.e., subgroups that were pre-specified in the grant proposal were reported and no new subgroup analyses were introduced in the publications. Of all individual subgroups that could be identified in the included projects, we analyzed if they were pre-specified or a post-hoc finding.

Results: In 20 (25%) of the 79 included projects, publications were completely in agreement with the grant proposal. Of the 149 pre-specified subgroups, 46 (31%) were reported in the final report or scientific publications, and 143 of the 189 (76%) reported subgroups were based on post-hoc findings. For 77% of the subgroup analyses in the publications, there was no mention whether these were pre-specified or post-hoc. Justification for subgroup analysis and methods to study subgroups were rarely reported.

Conclusion: There is a large discrepancy between grant applications and final publications regarding subgroup analyses. Both non-reporting of pre-specified subgroup analyses and reporting of post-hoc subgroup analyses are common.