Article type
Abstract
Background
The Japan Council for Quality Health Care (JQ) has managed an evidence-based medicine–promoting project that works on the evaluation, certification, and dissemination of clinical practice guidelines (CPGs) developed in Japan as a guideline clearinghouse. Guidelines that provide the most appropriate recommendation play an important role in supporting shared decision-making between patients and medical practitioners, but the quality of CPGs is not well understood.
Objectives
To clarify the trend and methodological quality of CPGs developed in Japan.
Methods
After searching and screening with exclusion criteria, we evaluated identified CPGs from the aspect of methodological quality using Appraisal of Guidelines for Research & Evaluation II Instrument (AGREE II) by a guideline evaluation expert committee. Each guideline was evaluated by 4 expert members, and CPG evaluation conferences were held to facilitate consensus building among members. We adopted the second scoring after the conferences and calculated the mean scores of domains and items of AGREE II.
Results
A total of 882 guidelines were evaluated by the AGREE II from September 2011 to October 2023. The mean scores of each AGREE II domain (0-100%) were as follows: Scope and Purpose, 70.2%; Stakeholder Involvement, 56.3%; Rigor of Development, 55.4%; Clarity of Presentation, 68.8%; Applicability, 47.5%; Editorial Independence, 55.6%; and Overall Guideline Assessment, 59.8%. The low-scoring items in the mean scores of Rigor of Development (<4.0, range 1-7) were as follows: item 8 (evidence selection criteria), item 9 (strengths and limitations of the evidence), and item 13 (external review).
Conclusions
This study indicates that Japanese CPGs have some tasks regarding description of evidence selection criteria, strengths and limitations of the evidence, and external review. As evaluation reporting leaves much to be desired in other items, it is necessary to utilize the evaluation results effectively for quality improvement. We are preparing to hold the meeting with the guideline development group (GDG) and to support the GDG individually.
The Japan Council for Quality Health Care (JQ) has managed an evidence-based medicine–promoting project that works on the evaluation, certification, and dissemination of clinical practice guidelines (CPGs) developed in Japan as a guideline clearinghouse. Guidelines that provide the most appropriate recommendation play an important role in supporting shared decision-making between patients and medical practitioners, but the quality of CPGs is not well understood.
Objectives
To clarify the trend and methodological quality of CPGs developed in Japan.
Methods
After searching and screening with exclusion criteria, we evaluated identified CPGs from the aspect of methodological quality using Appraisal of Guidelines for Research & Evaluation II Instrument (AGREE II) by a guideline evaluation expert committee. Each guideline was evaluated by 4 expert members, and CPG evaluation conferences were held to facilitate consensus building among members. We adopted the second scoring after the conferences and calculated the mean scores of domains and items of AGREE II.
Results
A total of 882 guidelines were evaluated by the AGREE II from September 2011 to October 2023. The mean scores of each AGREE II domain (0-100%) were as follows: Scope and Purpose, 70.2%; Stakeholder Involvement, 56.3%; Rigor of Development, 55.4%; Clarity of Presentation, 68.8%; Applicability, 47.5%; Editorial Independence, 55.6%; and Overall Guideline Assessment, 59.8%. The low-scoring items in the mean scores of Rigor of Development (<4.0, range 1-7) were as follows: item 8 (evidence selection criteria), item 9 (strengths and limitations of the evidence), and item 13 (external review).
Conclusions
This study indicates that Japanese CPGs have some tasks regarding description of evidence selection criteria, strengths and limitations of the evidence, and external review. As evaluation reporting leaves much to be desired in other items, it is necessary to utilize the evaluation results effectively for quality improvement. We are preparing to hold the meeting with the guideline development group (GDG) and to support the GDG individually.