Reporting and application of minimally important differences in randomized controlled trials evaluating patient reported outcomes: a systematic survey

Oral
2016 Seoul
Devji T1, Carrasco-Labra A2, Bhatt M1, Evaniew N3, Florez ID4, Montoya L5, Perera S1, Quach K1, Riva JJ6, Siemieniuk R7, Sivanand A8, Vercammen K9, Zeeratkar D1, Johnston B10, Cook DJ11, Guyatt G11, Ebrahim S12
1Department of Clinical Epidemiology & Biostatistics, McMaster University, Canada
2Department of Clinical Epidemiology & Biostatistics, McMaster University, Canada; Evidence-Based Dentistry Unit, Faculty of Dentistry, University of Chile, Santiago, Chile
3Department of Clinical Epidemiology & Biostatistics, McMaster University; Division of Orthopaedic Surgery, McMaster University, Canada
4Department of Clinical Epidemiology & Biostatistics, McMaster University, Canada; Department of Pediatrics, University of Antioquia, Medellin, Columbia
5Krembil Research Institute, TWH Orthopaedic Surgery Department, University Health Network, Canada
6Department of Clinical Epidemiology & Biostatistics, McMaster University; Department of Family Medicine, McMaster University, Canada
7Department of Clinical Epidemiology & Biostatistics, McMaster University; Department of Medicine, University of Toronto, Canada
8Department of Medicine, University of Toronto, Canada
9Faculty of Health Sciences, McMaster University, Canada
10Systematic Overview through Advancing Research Technology (SORT) program, Hospital for Sick Children Research Institute, Peter Gilgan Centre for Research and Learning, Canada
11Department of Medicine and Clinical Epidemiology & Biostatistics, McMaster University, Canada
12Department of Clinical Epidemiology & Biostatistics, McMaster University; Systematic Overview through Advancing Research Technology (SORT) program, Hospital for Sick Children Research Institute, Peter Gilgan Centre for Research and Learning, Canada

Background:

Despite the increasing use of patient reported outcomes (PROs) in randomized controlled trials (RCTs), interpreting treatment effects (trivial, small but important, or large) remains a challenge. The minimal important difference (MID) provides a measure of the smallest change in a PRO that patients would perceive as important, and can facilitate interpretation of RCT results.

Objectives:

We conducted the first comprehensive systematic survey of published RCTs to determine the extent to which trialists use MIDs when evaluating the impact of interventions on PROs.

Methods:

We searched MEDLINE, Embase, CENTRAL and PsycINFO to identify RCTs published in 2014 that evaluate the impact of interventions on at least one PRO. Reviewers screened identified citations, reviewed full texts of potentially eligible articles, and extracted relevant data from eligible studies.

Results:

44/478 RCTs (9%) that reported on at least one PRO reported an MID. The 478 trials included 1312 PROs, of which only 75 (6%) reported an MID. Of these, 21 (28%) estimated the MID through anchor-based methods, 13 (17%) through distribution-based methods, 13 (17%) through a combination of these methods, 6 (8%) used an expert consensus approach, and it was unclear for 22 (29%) studies. Information regarding the method used to estimate the reported MID was primarily retrieved from referenced articles (94%). Most (n = 64, 85%) used an MID to interpret the magnitude of effects: 39 (61%) analyses involved a comparison of mean effects in relation to the MID, 19 (30%) examined the proportion of patients achieving an improvement greater than or equal to the MID, and no quantitative analysis was presented in 7 (11%) instances. In multivariable analysis, features associated with reporting an MID were publication in a general medical journal (odds ratio (OR) 4.04 (95% CI 1.12 to 14.5)), adequate allocation concealment (OR 2.25 (95% CI 1.12 to 4.56)), and blinding of outcome assessors (OR 2.17 (95% CI 1.02 to 4.65)).

Conclusions:

RCT authors whose outcomes include PROs seldom report MIDs. When they do, they usually use the MIDs to help interpret the magnitude of treatment effects.