Article type
Abstract
"Background
In 2023, the Australian Living Evidence Collaboration began the Living Evidence for Australian Pregnancy and Postnatal Care (LEAPP) project to update Australia’s national pregnancy and postnatal care guidelines. Guidelines rely on the integrity of research to ensure their recommendations are valid. Research integrity is a major international challenge, with concerns including spin, selective reporting, fabrication of results and fraudulent studies. Pregnancy is a particular area of concern, with high levels of potentially problematic studies observed by studies of the published literature. Guideline development methodologies do not routinely incorporate integrity checks, and the feasibility of doing so for large guidelines has not been investigated.
Objectives
To design and pilot a process to identify potentially problematic studies included in systematic reviews informing the LEAPP guidelines.
Methods
In phase 1, PubMed was searched for retractions, expressions of concern and corrections within the scope of the LEAPP guidelines. Records were tagged with topic categories. In phase 2, an adapted version of the Trustworthiness Screening Tool (TST) developed by Cochrane Pregnancy and Childbirth was piloted to assess the feasibility of assessing integrity at scale. Studies included in a systematic review on progesterone and cervical cerclage underpinning the LEAPP guidelines were assessed. TST items requiring contact with study authors were not used.
Results
In phase 1, 266 records were identified, of which 87 were within scope and 35 were retractions. The most frequent topic categories were lifestyle considerations, clinical assessments during pregnancy and postnatal care. In phase 2, 20 studies were assessed using the TST, of which 12 had possible concerns requiring further investigation (such as implausible baseline characteristics or zero loss to follow-up). Assessments required considerable time and judgement, and were challenging for researchers without relevant clinical expertise.
Conclusions
Monitoring retractions and expressions of concern is feasible, but identifies a small proportion of possibly problematic studies. Assessment of individual studies would require considerable time and expertise to address a large number of topics and investigate possible concerns, but could have substantial impact on the guideline.
This project does not engage consumers directly but aims to improve the reliability of evidence-based guidelines to improve care."
In 2023, the Australian Living Evidence Collaboration began the Living Evidence for Australian Pregnancy and Postnatal Care (LEAPP) project to update Australia’s national pregnancy and postnatal care guidelines. Guidelines rely on the integrity of research to ensure their recommendations are valid. Research integrity is a major international challenge, with concerns including spin, selective reporting, fabrication of results and fraudulent studies. Pregnancy is a particular area of concern, with high levels of potentially problematic studies observed by studies of the published literature. Guideline development methodologies do not routinely incorporate integrity checks, and the feasibility of doing so for large guidelines has not been investigated.
Objectives
To design and pilot a process to identify potentially problematic studies included in systematic reviews informing the LEAPP guidelines.
Methods
In phase 1, PubMed was searched for retractions, expressions of concern and corrections within the scope of the LEAPP guidelines. Records were tagged with topic categories. In phase 2, an adapted version of the Trustworthiness Screening Tool (TST) developed by Cochrane Pregnancy and Childbirth was piloted to assess the feasibility of assessing integrity at scale. Studies included in a systematic review on progesterone and cervical cerclage underpinning the LEAPP guidelines were assessed. TST items requiring contact with study authors were not used.
Results
In phase 1, 266 records were identified, of which 87 were within scope and 35 were retractions. The most frequent topic categories were lifestyle considerations, clinical assessments during pregnancy and postnatal care. In phase 2, 20 studies were assessed using the TST, of which 12 had possible concerns requiring further investigation (such as implausible baseline characteristics or zero loss to follow-up). Assessments required considerable time and judgement, and were challenging for researchers without relevant clinical expertise.
Conclusions
Monitoring retractions and expressions of concern is feasible, but identifies a small proportion of possibly problematic studies. Assessment of individual studies would require considerable time and expertise to address a large number of topics and investigate possible concerns, but could have substantial impact on the guideline.
This project does not engage consumers directly but aims to improve the reliability of evidence-based guidelines to improve care."