Challenges for formulary recommendations: limita­tions of outcome data reported in systematic reviews

Article type
Authors
Sullivan S, O’Grady T, Pagotto S
Abstract
Background: The Common Drug Review (CDR) conducts systematic reviews (SRs) to support evidence-based Canadian formulary listing recommendations (FLRs). Studies evaluating new drugs are sometimes limited in the types of outcomes they evaluate, creating challenges for health policy decision makers. Objectives: To assess limitations of clinical trial outcomes reported in studies included in CDR SRs and to evaluate relevance to evidence-based FLRs. Methods: Outcomes data included in SRs were characterized. Additional context CDR provided in SRs on outcomes was identified. Frequency and type of outcome limitations noted in FLRs was considered. Results: Between 2003 and 2008, CDR conducted 111 SRs leading to FLRs. Twenty-seven per cent (30/111) SRs included a study in which the primary outcome was mortality or serious morbidity (16/111), harms (10/111) or quality of life (QoL) (5/111). Fifty-six per cent (62/111) SRs had no efficacy data on mortality or serious morbidity; 34% (38/111) SRs had no QoL data. Eighteen per cent (20/111) SRs had neither. SRs provided additional context on a total of 214 study endpoints. Fifty-three per cent (113/214) of scales and surrogate outcomes CDR reviewed had limitations (e.g. never validated; problems with reliability or validity; no known minimal clinically important difference). Forty-four per cent (49/111) FLRs identified study outcome limitations (e.g. no or unclear clinical significance of differences; no data on clinical outcomes) or the importance of having clinical outcome data. Forty-one per cent (46/111) of FLRs noted the absence or presence of QoL data. Conclusions: Limitations associated with clinical trial outcomes create challenges for policy makers who evaluate comparative effectiveness of new drugs. Given the lack of clinical outcome data in CDR SRs, further development of methodology that incorporates critical appraisal of scale and surrogate endpoints into SRs is required. Most CDR SRs include only RCTs. Consideration should be given to identifying best available evidence reporting clinical outcomes and developing rigorous methods of incorporating non-RCT evidence into CDR SRs.