Grading of Recommendations Assessment, Development and Evaluation (GRADE) Ontology Project

Poster
2024 Prague [Global Evidence Summit]
Dehnbostel J1, Whaley P2, Alper B1, Sayfi S3, Li S4, Bracchlione J5, Lehmann H6, Keshavarz H7, Mokrane S8, Dalsbo T9, Shahin K1, Wilkins K10
1Scientific Knowledge Accelerator Foundation, Franklin, North Carolina, United States
2Evidence Based Toxicology Collaboration, Lancaster, Lancashire, UK; Institute for Evidence-Based Toxicology, Lancaster, Lancashire, UK
3Department of Health Research Methods, Evidence and Impact, Michael G. DeGroote Cochrane Canada and GRADE Centres, McMaster University, Hamilton, Ontario, Canada
4Department of Endocrinology and Metabolism, MAGIC China Centre, West China Hospital of Sichuan University, Chengdu, Sichuan, China
5Iberoamerican Cochrane Centre, Institut de Recerca Sant Pau (IR Sant Pau), CIBERESP, Barcelona, Catalonia, Spain; Interdisciplinary Centre for Health Studies (CIESAL) Universidad de Valparaiso, Viña del Mar, Valparaiso, Chile
6Scientific Knowledge Accelerator Foundation, Franklin, North Carolina, United States; Johns Hopkins University, Baltimore, Maryland, United States
7Department of Family Medicine, McMaster University, Burlington, Ontario, Canada
8Department of Primary Care, Faculty of Medicine, Université Libre de Bruxelles, Brussels, Brussels, Belgium; Belgian Working Group development of primary care guidelines(WOREL), Wilrijk, Antwerp, Belgium
9STAMI - The National Institute of Occupational Health in Norway, Oslo, Oslo, Norway
10National Institutes of Health, National Institute of Diabetes & Digestive & Kidney Diseases, NIH Common Elements Governance Committee, Bethesda, Maryland, United States; Uniformed Services University of the Health Sciences, Department of Preventive Medicine & Biostatistics, Bethesda, Maryland, United States

Background:

GRADE is a de facto methodological framework and terminology standard for certainty-of-evidence and evidence-to-decision judgements. The GRADE working group’s guidances have led to a heterogeneous understanding of key terms, reducing the computability of evidence summaries. Computer applications such as GRADEPro and MAGICapp require crisp and well-defined terms. Formalising the terminology of GRADE into a harmonised ontology will provide a technical infrastructure that will enhance the efficiency of creating and reusing GRADE data.

Objective:

Develop and maintain a standardized, computable vocabulary of terms, an ontology, necessary for the expression of GRADE certainty-of-evidence and evidence-to-decision judgements.

Methods:

Using the 13 step Health Evidence Knowledge Accelerator Scientific Evidence Code System (HEvKA/SEVCO) term definition protocol, a group of GRADE experts meet virtually each week in open public meetings to curate terms from published GRADE guidances, establish preferred and alternative terms for included concepts, define the concepts, and write guidance for application. The aforementioned steps require the collective feedback of many, and a globally agreed approach across a diverse community with complex opinions about what is important woven together to create a comprehensive consensus. Any dissenting comments or votes are discussed until agreement is reached. We anticipate that after publication of version 1 of the ontology, the group will maintain and support the use and further development of the ontology.

Results:

The first version of the GRADE Ontology consists of 89 terms covering certainty of evidence and evidence-to-decision judgements. Fifty-one experts from multiple countries have participated thus far using a democratic consensus protocol. Five terms have been finalised with an average of 3.8 voting rounds, and 51 total votes per term, with an average of 9.8 votes per final round that results in 100% agreement.

Conclusions:

We have successfully migrated the HEvKA/SEVCO process to another domain and achieved a remarkable level of consensus. Feasibility and stability of the process have been demonstrated. That success will lead to coherence across GRADE reports and their computability allowing results of the GRADE process to be widely disseminated. This project is open to everyone, please join the project group using the following link: https://fevir.net/resources/Project/111563.