Article type
Year
Abstract
Background: When randomized clinical trials are conducted, ethical and scientific principles postulate that there must be a reasonable expectation that a research question will be answered and contribute to general knowledge resulting in societal benefit. Failure to provide reasonably definitive answers to questions addressed represent waste of precious resources and ethical breach of contracts with patients who expect that definitive answers will help future patients. Two main factors are thought to be responsible for generating non-significant findings in clinical research: poor patient accrual and expectation bias or ‘optimism bias’ - an unwarranted belief in the efficacy of new therapies (Lancet 2006; 367:449). Objectives: Our objective is to determine which of the two factors, poor patient accrual or expectation bias, is the main culprit for producing inconclusive clinical research results. Methods: We extracted data from the original protocols and publications related to 413 trials conducted by the four NCI-sponsored Cooperative Groups. We compared a prior effect size used in the power calculation for the trial’s pre-defined primary outcome (expected differences) with those observed in the final reports. We also extracted data on the planned vs. actual patient accrual. The question whether ‘negative’/inconclusive, i.e. statistically non-significant results, were due to poor accrual or due to ‘optimism’ bias was addressed. Results: Data on primary outcomes were available for 315 out of 413 studies. Seventy per cent of the studies had non-significant results (222/315). The vast majority of these trials were inconclusive, i.e. 95% CI of their treatments had crossed both limits of equivalence. Overall, the investigators enrolled more patients than actually planned [Nplanned: median: 220, mean: 339, range: 40–5200; Nactual: median: 249, mean: 373, range: 40–4406; p = 0.2189]. However, the expected effect size was considerably greater than actually observed differences in treatment effects: [Expected: mean = 43% (range: 19% to 150%) vs. Observed: mean = -0.02% (range: -119% to 79%; p = 0.000)]. Conclusions: Unrealistic expectations in treatment effects appear to be a major culprit for the continuous generation of large numbers of ‘negative’/inconclusive results.