Presentation of continuous outcomes in meta-analysis: a survey of clinicians’ understanding and preferences

Tags: Poster
Johnston B1, Bandayrel K1, Friedrich J2, Akl EA3, da Costa BR4, Neumann I5, Adhikari N6, Alonso-Coello P7, Crawford M8, Mustafa RA9, Svendrovski A10, Thabane L5, Tikkinen KA5, Vandvik PO11, Guyatt GH5
1Hospital for Sick Children Research Institute, Toronto, Canada, 2Keenan Research Centre, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, Canada, 3Department of Internal Medicine, American University of Beirut, Beirut, Lebanon, 4Division of Clinical Epidemiology & Biostatistics, Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland, 5Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Canada, 6Sunnybrook Health Sciences Centre, Toronto, Canada, 7Iberoamerican Cochrane Centre. Institute of Biomedical Research (IIB Sant Pau) Barcelona, Spain, 8Department of Anesthesia and Pain Medicine, Hospital for Sick Children, Toronto, Canada, 9Department of Internal Medicine, University of Missouri-Kansas City School of Medicine, United States, 10Hospital for Sick Children, Toronto, Canada, 11Norwegian Knowledge Centre for the Health Services, Oslo, Norway

Background: When pooling results of trials addressing continuous outcome using different instruments to measure the same construct, authors typically report differences between intervention and control as a Standardized Mean Difference (SMD). Recently, authors have proposed alternative summary estimates that they postulate clinicians’ will more easily interpret than SMD (1–4). The GRADE Working Group recently provided an overview of methods for presenting pooled continuous data (5). Thus far, claims of improved understanding with allegedly clinician-friendly presentations are supported only by anecdotes.

Objectives: To determine clinicians’ understanding and perceptions of six approaches (SMD, Minimal Important Difference Units, Natural Units, Relative Risk, Risk Difference and Ratio of Means) to the presentation of continuous outcomes from meta-analyses.

Methods: We invited 201 staff, residents, and trainees in family medicine and internal medicine academic programs in Canada, Switzerland and Lebanon to participate. Participants received paper-based self-administered surveys presenting summary estimates of hypothetical interventions versus placebo for chronic pain, with results demonstrating either a small effect or large effect for each of the six presentation approaches. We asked six questions addressing understanding and six questions addressing preferences. We randomized participants to size of effect and order.

Results: 188 clinicians responded (Table 1), with 175 providing completed surveys (87% response rate). Risk Difference was the approach best understood by clinicians, followed by the Ratio of Means and Relative Risk (Table 2). Clinicians generally found dichotomous presentation of continuous outcomes (Relative Risk; Risk Difference) very useful, and other approaches less useful (Table 3).

Conclusions: Clinicians best understood continuous outcomes when presented as dichotomies (relative and absolute risk differences) and found these presentations most useful. Presenting results as SMD, the longest standing and most widely used approach, was poorly understood and not perceived as useful. Data collection is ongoing and full results will be available at the colloquium.