Possible Bias in Selecting Interventions for Consumers Based on Effect Estimates

Article type
Year
Authors
Hedges L, Ramirez G, Sogolow E
Abstract
Introduction: It is important to offer guidance to consumers about what interventions (from a larger set of intervention research) work best to reduce sex- and drug-related risk associated with HIV transmission. If the effect size parameters were known without uncertainty, selecting the intervention with the largest effect size parameter would be an acceptable strategy. Unfortunately, effect parameters are subject to sampling uncertainty.

Objectives: The purpose of this paper is to investigate the probability that selecting the most effective intervention (that with the largest effect size parameter) introduces bias that is a function of using the observed effect estimate since the most effective intervention is selected based on that estimate.

Methods: We have developed a statistical model that permits us to compute the probability of correctly selecting the largest effect from a group of estimates and to compute the bias in estimates selected. This model depends on the number of studies ; available, the sampling error variance of the estimates, and the variation between studies in the effect parameters associated , with different interventions. For example, when the most effective intervention is selected from five that are equally effective (have equal effect size parameters), selection of the largest overestimates the effect by about one standard error of estimate. When selecting from among 20 equally effective interventions, the overestimation is about 1.8 standard errors of estimation. Data: We will use data from CDC's Compendium of HIV Prevention Interventions with Evidence of Effectiveness to provide reasonable values for exploration of the performance of selection procedures and bias in estimation.

Results: Our results will be used to explore intervention selection bias. We will discuss alternative strategies for addressing consumers' needs for science-based interventions that can be expected to reduce risk of HIV disease.

Discussion: