Article type
Abstract
Background: The integrity of evidence synthesis research is crucial for the credibility and reliability of findings. Systematic reviews, the gold standard evidence, enable decision-makers to make evidence-based decisions based on the best available evidence. Evidence gap maps (EGMs), a visual tool presenting the state of systematic reviews on a particular topic, enhance access to high-quality evidence, and show where the evidence lies and where little or poor-quality evidence exists. This presentation reports on trends in the quality of reviews included on the refractive error EGM (figure 1), between 2015 and 2022. It discusses the limitations of these reviews and presents solutions to ensure the integrity of the research.
Methods: The EGM approach draws on the principles of systematic reviews. Following a comprehensive search of relevant databases, we sift and extract data from all relevant reviews. Critical appraisal is conducted by two reviewers independently using Supported the Use of Research Evidence checklist attributing an overall confidence level: high, medium, and low.
Results: Between 2015 and 2022, the number of reviews increased from 24 to 187. These are categorised into low (n=87), medium (n=78) and high (n=23) confidence levels. A significant increase was observed in low (n=70), and medium (n=65) confidence reviews compared to those with high confidence (n=3). Most of low (n=47/87), and medium (n=47/78) confidence reviews provide conclusive answers, unlike most high confidence reviews (n=10/23) that found weak evidence. The low and medium confidence levels was mainly due to methodological limitations or inadequate reporting on research methods. Most reviews are based on English-only, peer-reviewed studies. Despite assessing the risk of bias in primary studies, low confidence reviews did not consider this in their meta-analysis or narrative synthesis, nor did they reflect it in their conclusions.
Conclusions: Research integrity is crucial for the validity and reliability of systematic review findings. There is a need for researchers to adhere standard methodological guidelines, engage key stakeholders, maintain transparency, evaluate the certainty/quality of included evidence, and present valid conclusions, while acknowledging the methodological constraints of the data and synthesis process.
Methods: The EGM approach draws on the principles of systematic reviews. Following a comprehensive search of relevant databases, we sift and extract data from all relevant reviews. Critical appraisal is conducted by two reviewers independently using Supported the Use of Research Evidence checklist attributing an overall confidence level: high, medium, and low.
Results: Between 2015 and 2022, the number of reviews increased from 24 to 187. These are categorised into low (n=87), medium (n=78) and high (n=23) confidence levels. A significant increase was observed in low (n=70), and medium (n=65) confidence reviews compared to those with high confidence (n=3). Most of low (n=47/87), and medium (n=47/78) confidence reviews provide conclusive answers, unlike most high confidence reviews (n=10/23) that found weak evidence. The low and medium confidence levels was mainly due to methodological limitations or inadequate reporting on research methods. Most reviews are based on English-only, peer-reviewed studies. Despite assessing the risk of bias in primary studies, low confidence reviews did not consider this in their meta-analysis or narrative synthesis, nor did they reflect it in their conclusions.
Conclusions: Research integrity is crucial for the validity and reliability of systematic review findings. There is a need for researchers to adhere standard methodological guidelines, engage key stakeholders, maintain transparency, evaluate the certainty/quality of included evidence, and present valid conclusions, while acknowledging the methodological constraints of the data and synthesis process.