Evaluating both benefits and harms of interventions: the need for equality

Tags: Oral
Craig J, Irwig L

Abstract: To make informed decisions about interventions, patients and healthcare providers should know about the range of benefits and harms possible and how likely each one is to occur with and without the intervention. Observation of the reporting of benefits and harms in randomized controlled trials and systematic reviews suggest more attention is given to the measurement and reporting of possible benefits of interventions than possible harms. The net effect of this practice is to bias clinical decision making in favour of interventions. This is likely to be increasingly problematic with decreasing marginal gains of newer interventions, when decisions are made closer to the net clinical benefit threshold. To evaluate the quality of in the evaluation of harms and benefits in randomized controlled trials we chose 38 trials from five systematic reviews of interventions used in children with kidney disease: growth hormone, immunosuppressive drugs in children with nephrotic syndrome, and antibiotics and surgery in children with urinary tract infection. The quality and detail of reporting of harms was substantially inferior to the quality and detail of intervention-related benefits. Common failures were no mention of adverse effects (11%), and in those studies which did report harms: incomplete listing of possible harms (100%), no group allocation of children who developed adverse effects (18%), uncertainty about the number of children evaluated for each harm (68%), and uncertainty about the number of children who developed a harm (34%). In addition, the trials were clearly underpowered to detect clinically important differences in intervention-related harms, even if measurement and reporting had been complete. In comparison, all 38 trials provided sufficient details about the benefits of the interventions for meta-analyses to be done. We suggest a number of strategies to ensure authors of systematic reviews explicitly give equal weight to both effects of healthcare interventions (beneficial and harmful) so that consumer and provider decision making can be fully informed - * all of the possible clinically important benefits and harms of the healthcare intervention evaluated be listed at the protocol stage. * the quality of the trial design and reporting of harms be considered separately for harms and interventions, and if different, be reported separately * the magnitude (and uncertainty) of all protocol-listed benefits and harms be given in the results section of the review, even when no data are available.