Evaluating priority-setting approaches: tools for a Cochrane Methodology Review

Article type
Authors
Nasser M1, Welch V2, Ueffing E2, Tugwell P2, Chalkidou K3, Sawicki P4, Crowe S5
1Peninsula Dental School, University of plymouth, Plymouth, UK
2Campbell and Cochrane Equity Methods Group, Institute of Population Health, University of Ottawa, Canada
3National Institute for Health and Clinical Excellence, London, UK
4Institute of Health Economics and Clinical Epidemiology, University of Cologne, Cologne, Germany
5James Lind Alliance Secretariat, James Lind Initiative, Oxford, UK
Abstract
Background: There is a continuing interest in defining priority topics and setting a research agenda for systematic reviews inside and outside of the Cochrane Collaboration. However, there is no consensus on the best methods of priority-setting. Therefore, we are conducting a Cochrane methodology review to address this gap. There is no agreement on the tools to critically appraise the quality (transparency, effectiveness, rigor, etc.) of conducting and reporting of the priority setting methods.

Objectives: We aimed to develop critical appraisal checklists to evaluate the quality of the priority setting methods in the Cochrane methodology review.

Methods: We did a search for studies on the criteria for good or successful priority-setting methods and studies on how equity can be incorporated in priority-setting methods. The results of the literature search were discussed in two workshops at previous Colloquia (Singapore and Keystone). The results of the discussions along with the consultation of experts in the field were used to develop two checklists for the Cochrane methodology review.

Results: We developed two checklists, one of the checklists with eight questions to evaluate the transparency and rigour of the process of conducting and reporting of the priority setting methods. We also developed an equity lens with 17 questions across the different steps of a prioritization process.

Conclusions: The checklists could be a potentially helpful tool to evaluate primary studies for our Cochrane methodology review and might even help primary researchers in planning and developing priority-setting methods. We will be working further to incorporate other aspects of critically appraising priority setting methods, especially differentiating priority setting methods with better outcomes like efficiency/value of money, and impact. The project also provides guidance for authors of methodology reviews on methods in situations where there is no agreed critical appraisal tool.