Article type
Abstract
"Background
The volume of new scientific publications a healthcare provider needs to digest to deliver evidence-based care is overwhelming, particularly in primary care, where the scope is comprehensive. In low- and middle-income countries (LMICs), clinicians and policymakers find synthesized evidence in WHO guidance (but in 2019 alone, needed to scan 155 publications) and in electronic clinical decision support products (however subscription costs and well-resourced setting focus make these largely inaccessible). Over 20 years, we developed, implemented, and evaluated a comprehensive clinical decision support tool (CDST), Practical Approach to Care Kit (PACK). It currently supports primary care decision-making in South Africa, Brazil, Ethiopia and Indonesia, making latest evidence accessible at point-of-care.
Objectives
To develop efficient CDST creation and updating processes that make relevant, up-to-date evidence accessible to LMIC primary care policymakers and healthcare providers.
Methods and results
In collaboration with WHO’s Clinical Services and Systems Unit, BMJ’s Best Practice and LMIC healthcare stakeholders, we drew on synthesised evidence and end-user input to craft 3500 CDST recommendations covering over 500 symptoms and conditions. We regularly access Best Practice monograph change reports which flag updated content and scan WHO’s IRIS website for new publications to identify updates for PACK. To optimise these processes, we developed an artificial intelligence (AI) assistant utilising retrieval augmented generation (RAG), vectorised databases, automated web-scraping and large language model summarisation. More modernised technical approaches have improved evidence source curation and interrogation and data provenance. We are currently cross-validating the AI model against clinical editor outputs to determine safety across domains of scientific merit, risk of patient harm and inappropriate content generation. Preliminary analysis (prior to adoption of RAG) shows 90-94% sensitivity and positive predictive value compared to human intelligence.
Conclusion
Updating mechanisms that leverage high-income country evidence synthesis processes and WHO publication events provide a feasible approach to maintaining PACK’s comprehensive CDST content. It appears that an AI assistant model has potential to augment our ability to keep up with the churn of evidence. This approach could help address inequities in access to evidence, ensuring LMIC healthcare policymakers, providers and patients receive latest, relevant evidence to support primary care.
"
The volume of new scientific publications a healthcare provider needs to digest to deliver evidence-based care is overwhelming, particularly in primary care, where the scope is comprehensive. In low- and middle-income countries (LMICs), clinicians and policymakers find synthesized evidence in WHO guidance (but in 2019 alone, needed to scan 155 publications) and in electronic clinical decision support products (however subscription costs and well-resourced setting focus make these largely inaccessible). Over 20 years, we developed, implemented, and evaluated a comprehensive clinical decision support tool (CDST), Practical Approach to Care Kit (PACK). It currently supports primary care decision-making in South Africa, Brazil, Ethiopia and Indonesia, making latest evidence accessible at point-of-care.
Objectives
To develop efficient CDST creation and updating processes that make relevant, up-to-date evidence accessible to LMIC primary care policymakers and healthcare providers.
Methods and results
In collaboration with WHO’s Clinical Services and Systems Unit, BMJ’s Best Practice and LMIC healthcare stakeholders, we drew on synthesised evidence and end-user input to craft 3500 CDST recommendations covering over 500 symptoms and conditions. We regularly access Best Practice monograph change reports which flag updated content and scan WHO’s IRIS website for new publications to identify updates for PACK. To optimise these processes, we developed an artificial intelligence (AI) assistant utilising retrieval augmented generation (RAG), vectorised databases, automated web-scraping and large language model summarisation. More modernised technical approaches have improved evidence source curation and interrogation and data provenance. We are currently cross-validating the AI model against clinical editor outputs to determine safety across domains of scientific merit, risk of patient harm and inappropriate content generation. Preliminary analysis (prior to adoption of RAG) shows 90-94% sensitivity and positive predictive value compared to human intelligence.
Conclusion
Updating mechanisms that leverage high-income country evidence synthesis processes and WHO publication events provide a feasible approach to maintaining PACK’s comprehensive CDST content. It appears that an AI assistant model has potential to augment our ability to keep up with the churn of evidence. This approach could help address inequities in access to evidence, ensuring LMIC healthcare policymakers, providers and patients receive latest, relevant evidence to support primary care.
"