Article type
Abstract
"Background: Understanding the alignment and contributions of scientific research to the Sustainable Development Goals (SDGs) is essential for guiding global progress toward these critical targets. Several SDG mapping approaches have been proposed and tested by organisations and researchers but have not produced consistent mapping results. AI-powered search functionality, with its capacity to analyse vast datasets and identify patterns, has been presented as an innovative mechanism for the tracking, analysis and reporting of SDG datasets and insights to assess progress towards targets and facilitate informed data-driven decisions.
Objective: To assess the reliability and utility of AI in mapping evidence syntheses (publications) to the SDGs.
Methods: Scopus, Dimensions and Altmetric have built their own classification systems to map research outputs to SDGs using unique search queries and machine learning algorithms. Using the various classification filters across the three databases, this study reviewed and compared the alignment of 204 evidence syntheses published in JBI Evidence Synthesis from 2016-2022 to SDG 3, a dataset previously mapped to SDG 3 in a pilot study by the author team using a manual approach.
Results: Results highlighted significant discrepancies in the publications mapped to SDG 3 both across the three databases employing AI and with the pilot study. Complete concurrence occurred for only 52% of publications (n=14 mapped to SDG 3 and n=92 not mapped, out of 204) across the three databases and pilot study. Of the remaining 98 evidence syntheses, 12% (n=24) were mapped by three, 15% (n=31) were mapped by two and 21% (n=43) were mapped by just one database or pilot study.
Conclusions: The findings of this study emphasize the need for standardised and transparent SDG mapping methods as the 2030 agenda deadline nears and progress on numerous targets lags. Identifying trustworthy evidence across the ecosystem is critical, but AI's trustworthiness to contribute to this is yet to be established with a level of confidence. It cautions the scientific community, policymakers, funders, and other stakeholders about the importance of comprehensive evaluation when mapping publications to SDGs.
"
Objective: To assess the reliability and utility of AI in mapping evidence syntheses (publications) to the SDGs.
Methods: Scopus, Dimensions and Altmetric have built their own classification systems to map research outputs to SDGs using unique search queries and machine learning algorithms. Using the various classification filters across the three databases, this study reviewed and compared the alignment of 204 evidence syntheses published in JBI Evidence Synthesis from 2016-2022 to SDG 3, a dataset previously mapped to SDG 3 in a pilot study by the author team using a manual approach.
Results: Results highlighted significant discrepancies in the publications mapped to SDG 3 both across the three databases employing AI and with the pilot study. Complete concurrence occurred for only 52% of publications (n=14 mapped to SDG 3 and n=92 not mapped, out of 204) across the three databases and pilot study. Of the remaining 98 evidence syntheses, 12% (n=24) were mapped by three, 15% (n=31) were mapped by two and 21% (n=43) were mapped by just one database or pilot study.
Conclusions: The findings of this study emphasize the need for standardised and transparent SDG mapping methods as the 2030 agenda deadline nears and progress on numerous targets lags. Identifying trustworthy evidence across the ecosystem is critical, but AI's trustworthiness to contribute to this is yet to be established with a level of confidence. It cautions the scientific community, policymakers, funders, and other stakeholders about the importance of comprehensive evaluation when mapping publications to SDGs.
"