Article type
Year
Abstract
Background: the National Collaborating Centre for Methods and Tools (NCCMT) provides leadership and expertise in evidence-informed decision making to organizations that deliver or make decisions about public health services. We share knowledge broadly about what works in public health and provide essential training and mentoring to support capacity development for evidence-informed decision making (EIDM). The result is that practitioners make informed decisions that support individual Canadians and their communities to achieve optimal health.
Objectives: to develop an evaluation framework that effectively illustrates the impact of NCCMT in relation to the reach of the centre’s activities, the quality of available products and services, and the impact on public health individuals and organizations.
Methods: NCCMT contracted an expert in programme evaluation to assist in the development of a revised evaluation framework that would better capture: reach, quality and impact. The contractor reviewed a number of internal documents: vision, mission, goals; annual workplans; annual reports (submitted to funder), and previous evaluation reports. A review of programme evaluation literature was also conducted. An inductive approach was used to identify evaluation themes.
Results: We developed a revised evaluation framework, influenced by theory of change, which is now being implemented at NCCMT. The evaluation framework encompasses four stages, each with specific outcome indicators identified, that progress from creation of products and services, to knowledge and awareness of these resources, to engagement and capacity development, and finally to policy and practice changes that become embedded within organizational structures. Specific outcome indicators include: confidence, knowledge, skills, organizational supports for EIDM, using evidence in decision making, conducting rapid reviews; critical appraisal of evidence, improved programmes, efficiency, engagement of workforce, and workforce competence for EIDM.
Conclusions: an evaluation framework informed by theory of change resulted in significant streamlining of the centre’s evaluation activities, and has provided clearer direction with respect to the purpose of evaluation, as well as enhanced ability to capture the centre’s reach, quality and impact. This presentation will not only describe the model stages and outcome indicators, but also highlights from the 2018 to 2019 fiscal year evaluation using the framework.
Patient or healthcare consumer involvement: not applicable
Objectives: to develop an evaluation framework that effectively illustrates the impact of NCCMT in relation to the reach of the centre’s activities, the quality of available products and services, and the impact on public health individuals and organizations.
Methods: NCCMT contracted an expert in programme evaluation to assist in the development of a revised evaluation framework that would better capture: reach, quality and impact. The contractor reviewed a number of internal documents: vision, mission, goals; annual workplans; annual reports (submitted to funder), and previous evaluation reports. A review of programme evaluation literature was also conducted. An inductive approach was used to identify evaluation themes.
Results: We developed a revised evaluation framework, influenced by theory of change, which is now being implemented at NCCMT. The evaluation framework encompasses four stages, each with specific outcome indicators identified, that progress from creation of products and services, to knowledge and awareness of these resources, to engagement and capacity development, and finally to policy and practice changes that become embedded within organizational structures. Specific outcome indicators include: confidence, knowledge, skills, organizational supports for EIDM, using evidence in decision making, conducting rapid reviews; critical appraisal of evidence, improved programmes, efficiency, engagement of workforce, and workforce competence for EIDM.
Conclusions: an evaluation framework informed by theory of change resulted in significant streamlining of the centre’s evaluation activities, and has provided clearer direction with respect to the purpose of evaluation, as well as enhanced ability to capture the centre’s reach, quality and impact. This presentation will not only describe the model stages and outcome indicators, but also highlights from the 2018 to 2019 fiscal year evaluation using the framework.
Patient or healthcare consumer involvement: not applicable