A systematic review of the quality and types of instruments used to assess KTE implementation and impact

Article type
Authors
Irvin E1, Cole D2, Keown K1, Kramer D3, Brenneman Gibson J1, Kazman Kohn M4, Mahood Q1, Slack T1, Amick III B1, Phipps D5, Garcia J3, Morassaei S1
1Institute for Work & Health, Canada
2University of Toronto, Canada
3University of Waterloo, Canada
4St Michaelás Hospital, Canada
5York University, Canada
Abstract
Background: Knowledge transfer and exchange (KTE) is a burgeoning practice at research organizations worldwide. However the effective- ness of KTE practices has not been routinely/consistently evaluated. The need for such evaluation has been voiced (Lavis et al, 2003; Eccles et al, 2005). One potential reason for the lack of evaluation is the paucity of tools and methods for evaluation across various disciplines.

Objectives: To answer the following question: Are there reliable, valid and/or useful instruments to apply in the assessment of KTE implementation and its impact?

Methods: A team of researchers and stakeholders followed a systematic review process developed by the Institute for Work & Health using a Cochrane-like approach and a descriptive synthesis. The basic review steps include: formulation of research question and search terms; decision-maker/stakeholder meetings; literature search; relevance review; quality appraisal; data extraction; evidence synthesis; decision-maker/stakeholder key message generation; and report preparation.

Results: The search was conducted in MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, CAB Abstracts, LISA, Social Science Abstracts, and Business Source Premier. The search resulted in 9998 unique titles and abstracts (see Figure 1). We found 54 quantitative articles that described instruments for the evaluation of KTE approaches. Data extraction focused on instruments and measurement properties.

Conclusions: Our synthesis of 54 quantitative studies revealed a variety of instruments used to evaluate KTE approaches. Overall, the description of the measurement properties was not consistent and, often, details of measurement properties were lacking or missing. However, a subset of articles, provided descriptions of instrument development and measurement properties and show promise as potentially useful tools in the evaluation of KTE approaches. Key messages from this review include the need to continue instrument development work and to clearly present the measurement properties of the instruments used.