Background: The utility and external validity of research evidence is emerging as an issue in need of more scrutiny in the reporting of studies and systematic reviews purporting to be policy relevant. Objective: To assess the external validity of research evidence in a systematic review of the health impacts of housing improvement. Methods: We adapted an existing tool (Green & Glasgow, 2009) to assess the external validity of studies reported in a recent systematic review. The tool focussed on the transferability of the intervention and outcomes to other settings and populations, and the costs and implementation of the intervention. Results: There was considerable variation in the implementation of the intervention and the potential to benefit among the study sample. Some housing programmes were delivered according to need and others delivered across neighbourhoods regardless of individual need. Few studies reported details of variation in implementation, and sample sizes were often too small to merit sub-group analysis. A small number of studies reported intervention costs; three studies conducted an economic analysis comparing the intervention costs with the relative improvements in health and quality of life among residents. Within the largest group of studies judged to be similar, i.e. warmth improvements, the direction of reported health impacts varied. The characteristics of the intervention, context, and study population emerged as a determinant of the reported health impacts. Discussion: Clearer reporting of the intervention, including cost and implementation, could promote the transferability of single studies to specific policy and practice contexts. The systematic review examined had a broad scope and there was extreme heterogeneity among the studies. In addition to variation in study methods, there was wide variation in the intervention, outcomes, study population and context. Further examination of this heterogeneity contributed to a refinement of the hypotheses around housing improvements and health impacts.