Generalizability of results from randomized trials; A systematic overview of possible approaches

Article type
Authors
Post P1, Guyatt G2
1Dutch Institute for Healthcare Improvement CBO, Utrecht, Netherlands
2Departments of Clinical Epidemiology & Biostatistics and Medicine, McMaster University, Hamilton, Ontario, Canada
Abstract
Background: Randomized controlled trials (RCTs) consititute the preferred evidence source for recommendations regarding the effect of treatment. Unfortunately, patients participating in RCTs frequently differ importantly from most patients seen in practice. Therefore, guideline developers must decide whether results are generalizable to the target population not represented in RCTs. Objective: To identify methods to help decide the circumstances under which the results from RCTs can be generalized to patients who were not represented in these trials. Methods: A systematic search in Medline, Embase and other sources was done to identify possible methods that help to decide whether results are generalizable. Results: A frequently recommended approach is that the trial population should be representative of the broad patient group. This approach implies that numerous exclusion criteria applied in trials would diminish the generalizability. To evaluate the extent of the generalizability, one examines the in- and exclusion criteria of trials and infers from these whether the trial population was sufficiently representative. Other authors suggest that, because they include a broader range of patients, observational studies constitute the optimal source of evidence if no RCTs have directly addressed the target population. Another approach suggests applying the results of RCTs to patients in practice unless there is a compelling reason to believe the results would differ substantially as a function of particular characteristics. This approach is supported by empirical evidence that, in general, treatment effects seldom differ to an important extent across subgroups of patients. Conclusion: We propose this last approach, focusing on RCTs unless there is compelling reason not to do so. Compelling reasons will most often be found with respect to issues of rare adverse effects, for which observational studies are likely to provide the best estimates.