Article type
Year
Abstract
Background: Prognostic prediction models require a specific approach for evaluation in a systematic review. The CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) provides guidance for data extraction from prediction model studies. The Prediction model Risk Of Bias ASsessment Tool (PROBAST) is a complementary tool for in-depth assessment of risk of bias (RoB) and applicability of such studies.
Objectives: To systematically summarize results and assess RoB and applicability of studies that developed and/or validated a regression model predicting patient presentation rate (PPR) or transfer to hospital rate (TTHR) at mass gatherings, using the CHARMS and PROBAST checklists.
Methods: According to 7 key items of the CHARMS checklist, we systematically searched and classified model development and model validation studies, and extracted predictors for PPR or TTHR from multivariable regression models. The PROBAST checklist was used to assess RoB and applicability in 4 domains (participants, predictors, outcome, and analysis). We implemented overall RoB and applicability judgement into the Grading of Recommendations Assessment, Development and Evaluation (GRADE) tool.
Results: We identified 13 prediction model development studies without validation and 3 external validation studies of existing models, comprising >1,700 mass gatherings. Main predictors of PPR and/or TTHR were accommodation (e.g. indoor vs outdoor), type of event (e.g. music concerts), and weather conditions (e.g. temperature)..
PROBAST domains that were most prone to bias were the method of analysis used and selection of participants (high RoB in 13 (81%) and 10 (62%) studies, respectively). Consequently, we judged overall RoB as ‘high’ for all included studies (Figure 1). Overall concerns for applicability were high in 12 studies (75%), mainly due to high concerns for selection of participants (9 studies, 56%) (Figure 2).
The initial GRADE level of certainty in the body of evidence was set at ‘high’. We downgraded with two levels due to the high overall RoB (-1) and concerns regarding applicability (GRADE domain ‘indirectness’, -1), ending up with ‘low’ certainty in the effect estimates.
Conclusions: The CHARMS and PROBAST checklists proved to be useful for data extraction and assessment of RoB and applicability in prediction modelling studies on medical usage at mass gatherings. As such, they are complementary with the GRADE evaluation of the body of evidence.
Patient or healthcare consumer involvement: This systematic review and the development and validation of a proper prediction model, is conducted in collaboration with the Relief Service at the Belgian Red Cross, which coordinates the preventive aid campaigns at mass gatherings in Flanders (Belgium). Regular meetings with central coordinators and representatives of local volunteers helped identifying strengths and weaknesses of the current databases and desired features of our own prediction model for medical usage rate at mass gatherings.
Objectives: To systematically summarize results and assess RoB and applicability of studies that developed and/or validated a regression model predicting patient presentation rate (PPR) or transfer to hospital rate (TTHR) at mass gatherings, using the CHARMS and PROBAST checklists.
Methods: According to 7 key items of the CHARMS checklist, we systematically searched and classified model development and model validation studies, and extracted predictors for PPR or TTHR from multivariable regression models. The PROBAST checklist was used to assess RoB and applicability in 4 domains (participants, predictors, outcome, and analysis). We implemented overall RoB and applicability judgement into the Grading of Recommendations Assessment, Development and Evaluation (GRADE) tool.
Results: We identified 13 prediction model development studies without validation and 3 external validation studies of existing models, comprising >1,700 mass gatherings. Main predictors of PPR and/or TTHR were accommodation (e.g. indoor vs outdoor), type of event (e.g. music concerts), and weather conditions (e.g. temperature)..
PROBAST domains that were most prone to bias were the method of analysis used and selection of participants (high RoB in 13 (81%) and 10 (62%) studies, respectively). Consequently, we judged overall RoB as ‘high’ for all included studies (Figure 1). Overall concerns for applicability were high in 12 studies (75%), mainly due to high concerns for selection of participants (9 studies, 56%) (Figure 2).
The initial GRADE level of certainty in the body of evidence was set at ‘high’. We downgraded with two levels due to the high overall RoB (-1) and concerns regarding applicability (GRADE domain ‘indirectness’, -1), ending up with ‘low’ certainty in the effect estimates.
Conclusions: The CHARMS and PROBAST checklists proved to be useful for data extraction and assessment of RoB and applicability in prediction modelling studies on medical usage at mass gatherings. As such, they are complementary with the GRADE evaluation of the body of evidence.
Patient or healthcare consumer involvement: This systematic review and the development and validation of a proper prediction model, is conducted in collaboration with the Relief Service at the Belgian Red Cross, which coordinates the preventive aid campaigns at mass gatherings in Flanders (Belgium). Regular meetings with central coordinators and representatives of local volunteers helped identifying strengths and weaknesses of the current databases and desired features of our own prediction model for medical usage rate at mass gatherings.