Background:

In recent decades there have been important and rapid changes in how evidence is identified, assessed and synthesised and also in methods of guideline development. The National Institute for Health and Care Excellence has developed guidelines for more than 15 years in the UK and demonstrated the importance of timely response to methodological developments in order to ensure robustness of its guideline recommendations. Developments are identified through current awareness and collaboration with various international organisations, especially Cochrane, GRADE and G-I-N.

Objectives:

To share our experience of introducing new methods in systematic reviews and guidelines.

Methods:

An evaluation of more than 15 years’ experience in evidence synthesis and guideline development, focusing on shared learning as methodologists and developers. Horizon scanning can help monitor and assess the potential for these.

Results:

New evidence synthesis methods are continually being developed in research and guideline developers need to be aware of these and be able to adapt and introduce new ways of working as appropriate. We present two detailed case studies.

GRADE: Practical aspects of introducing change in established work programmes: how methodological change can be introduced to improve the quality of guidelines without impacting on outputs. Problems included shifting from study-based to outcome based appraisal, defining outcomes and MIDs, tackling imprecision, differences between Cochrane reviews and guidelines, strong’ and ‘weak’ recommendations.

Network meta-

analysis:

Methodological aspects of introducing new analytical approaches in systematic reviews: when they are appropriate, quality assessment including application of GRADE, examining outputs for believability relative to expectations.

Conclusions:

The scale and pace of methodological development has increased over time. Systematic reviewers and guideline developers need to be able to identify these in a timely way and adapt methods as appropriate. When introducing change it is important to learn from other settings and organisations as well as sharing experiences and learning with others.