Background: Adjusted indirect comparison has been increasingly used in the assessment of healthcare interventions. We use evidence from three case studies to propose a radical idea that, under certain conditions, adjusted indirect comparison may provide a less biased estimate than direct comparison trials.

Methods: Both direct comparison and adjusted indirect comparison were used to compare bupropion with nicotine replacement therapy (NRT) for smoking cessation, risperidone with haloperidol for schizophrenia, and fluoxetine with imipramine for depression. The Cochrane Library and MEDLINE were searched to identify the relevant reviews and trials. We also conducted computer simulations to investigate bias in placebo-controlled trials and adjusted indirect comparisons.

Results: In all the cases, treatment effects of new drugs estimated by direct comparison trials tend to be greater than those by adjusted indirect comparisons. The discrepancy was nearly statistically significant in the case of bupropion vs NRT (p=0.055) and of risperidone vs haloperidol (p=0.058 for dropouts and p=0.015 for clinical improvement). It was not statistically significant in the third case of fluoxetine vs imipramine (p=0.36 for dropouts and p=0.091 for clinical improvement). The observed discrepancies could not be satisfactorily explained by the play of chance or by heterogeneity and bias in adjusted indirect comparison. This observation, along with analysis of possible systematic bias in randomized controlled trials, suggested that the indirect method had produced more valid results. Simulations reproduced the circumstances where this could occur.

Conclusions: Empirical evidence from three case studies indicates that adjusted indirect comparison may counter balance bias in head-to-head randomized trials under certain circumstances. This hypothesis needs to be examined systematically by further research.