What is the probability that results of meta-analyses will favor innovative treatments?

Article type
Authors
Djulbegovic B, Lacevic M, Macy T, Adams J, Lyman G
Abstract
Abstract: "What is the probability of a proposed new treatment being superior to established treatments?" in a randomized controlled trial (RCT) was the question that was previously asked in the literature (BMJ 1997;314:74). However, what is the probability that meta-analyses (MA) will find that innovative treatments are more successful has not been previously addressed. To address this issue more systematically we reviewed all MAs related to cancer in Database of Abstracts of Reviews of Effectiveness (DARE) in the Cochrane Library. The results of the each MA was rated on a six point scale (1-standard treatment highly preferred, 6-experimental treatment highly preferred). Of potentially 228 MAs identified in DARE database 45 MAs analyzed various aspects health care interventions related to cancer. A median evaluation score for all 45 MAs was 5 , ranging from a score of 1 (7%) to a score 6 (20% of MAs). Overall, we found that innovative treatments were favored in 69% (31/45) while standard treatments were preferred in 31% of MAs (p=0.011). Since we have previously shown that outcomes of trials could be violated as a function of funding source (Blood 1999;94:399a), we also analyzed results of MAs according to funding source. The data on funding were reported for 25% (11/45) MAs only. Seventy-three percent of these (8/11) were supported by public resources. All three MAs (100%) funded by industry favored innovative treatments. Among MAs funded by public resources, 25% (2/8) favored standard treatment, while 75% (6/ 8) favored the innovative treatment (p=0.157). Since we have recently shown that there is a predictable relationship between "the uncertainty principle", that is a fundamental scientific and ethical principle upon which RCTs are (Lancet, 2000, in press) founded, and outcome of these trials, we also postulate that ultimate results of MAs (that is, a research synthesis of multiple RCTs) may be predicted by "the uncertainty principle". According to this hypothesis, overall benefits of MAs , in the long run ,will result in similar proportions of studies favoring innovative or standard treatment, respectively among those studies funded by public resources; innovative treatments would be favored in those MAs funded by industry. While our preliminary results are consistent with this hypothesis, a larger sample size and complete funding data are needed to fully test it.