A survey of systematic reviews to identify items for assessing risk of bias in individual studies addressing values and preferences

Article type
Authors
Yepes-Nuñez JJ1, Zhang Y1, Alonso-Coello P2, Selva A3, Schunemann H4, Guyatt G5
1PhD candidate, Department of Clinical Epidemiology & Biostatistics at McMaster University, Canada
2Associate Research, Iberoamerican Cochrane Centre, Institute of Biomedical Research (IIB Sant Pau), Spain
3ssociate Research, Iberoamerican Cochrane Centre, Institute of Biomedical Research (IIB Sant Pau), Spain
4Professor and Chair, Department of Clinical Epidemiology & Biostatistics at McMaster University, Canada
5Distinguished Professor, Department of Clinical Epidemiology & Biostatistics at McMaster University, Canada
Abstract
Background: Use of systematic review methods to synthesize evidence addressing distributions of values and preferences of patients and general populations is becoming more frequent, but there is limited agreement on how to appraise the risk of bias (RoB) of values and preferences studies.
Objectives: To identify criteria for assessing RoB suggested by authors of systematic surveys of primary studies of values and preferences associated with specific health states.
Methods: We searched for reviews that addressed primary studies that used at least one of the following techniques for eliciting values and preferences:
1. choices patients make when presented with decision aids for management options;
2. primary studies that used direct choice methods of establishing values and preferences with techniques such as real life and hypothetical scenarios, probability trade-off; and
3. utilities or patient values from direct measurements such as the standard gamble (SG), time trade-off (TTO), visual analogue scales (VAS), rating scale, willingness trade-off (WTO) and willingness to pay (WTP).
Results: Our search strategy identified over 20,000 titles and abstracts; 250 were judged potentially eligible. Checking full text revealed 63 of the 250 to be potentially eligible (see Figure 1 for reasons for exclusion); 28 of the 63 did not report separate information about direct methods and associated RoB clearly (the focus of this project) and were excluded, leaving 35 eligible reviews. Preferences in the primary studies were measured several ways that included TTO (26%), SG (25%), VAS (12%), direct choice (9%), WTP (8%) and other less frequent methods were discrete choice experiment (2%), conjoint analysis (5%), WTO (1%), decision aid (3%), stated preference other (1%), and other types of non-standardized questionnaires (3%). We identified 31 items addressing RoB. Table 1 presents the items grouped by similar construct.
Conclusions: Authors of systematic reviews addressing values and preferences have identified over 30 items related to RoB. This survey provides groundwork for the creation of a RoB tool for assessing RoB in individual studies of values and preferences.