Objectives: To provide the review author with a background to the problem of outcome selection bias and how it might lead to misleading conclusions, to demonstrate how a review author might assess the likely presence of such bias in their review, and to present simple techniques for assessing the robustness of the meta-analysis to such bias.

Summary: Within-study selective reporting bias has been defined as the selection, on the basis of the results, of a subset of the analyses undertaken to be included in a study publication1. Sources of such reporting bias will be described, however, the workshop will focus on outcome selection bias. The effect of within-study selective reporting of outcomes will be demonstrated. Direct empirical evidence for the existence of outcome selection bias is accumulating2,3. In a meta-analysis it is often the case that a total number of k eligible studies are identified but only n report the data of interest. The review author needs to examine the remaining (k-n) studies to establish whether the outcome of interest has been collected but not reported. This should ideally involve contact with the original trialists which may result in missing data being made available or it may confirm that the outcome data were not recorded4. However, it is likely that in a subset of these studies, m (? k-n) say, no such information is forthcoming. It is important to assess the level of suspicion that selective non-reporting has occurred in these m studies. Methods for the identification of within-study selective reporting in a meta-analysis and an individual study will be described and illustrated using real examples. If the level of suspicion is high, a useful first stage is to undertake a sensitivity analysis to assess the robustness to extreme selective reporting. Two methods for such an analysis4,5 will be illustrated and compared using real examples. Participants will be encouraged to undertake such assessments for examples provided and to discuss the issues for their own reviews.

References
1. Hutton JL, Williamson PR. Bias in meta-analysis due to outcome variable selection within studies. Applied Statistics 2000; 49(3):359-70.
2. Chan A-W, Hróbjartsson A, Haar MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004; 291(20):2457-65.
3. Chan A-W, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institute of Health Research. CMAJ: Canadian Medical Association Journal 2004; 171(4):735-40.
4. Williamson PR, Gamble C. Identification and impact of outcome selection bias in meta-analysis. Statistics in Medicine
2005;24(10):1547-61.
5. Copas J, Jackson D. A bound for publication bias based on the fraction of unpublished studies. Biometrics 2004; 60(1):146-53.

Level of knowledge required to attend: intermediate.