Article type
Year
Abstract
Background: prognostic prediction models require a specific approach for evaluation in a systematic review. The recent Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) provides a framework for data extraction from and risk of bias assessment of such studies.
Objectives: to assess risk of bias of studies that developed and/or validated a regression model predicting patient presentation rate (PPR) or transfer to hospital rate (TTHR) at mass gatherings, using the CHARMS checklist.
Methods: we systematically searched for prediction modelling studies in six databases, classified included studies and extracted predictors for PPR or TTHR from multivariate regression models, based on the seven key issues pointed out in the first part of the CHARMS checklist. Next, we estimated risk of bias in 12 items, according to the second part of the CHARMS checklist. These items include general issues such as blinding of assessment and handling of missing data, but also specific issues for prediction modelling studies, such as method for selection of predictor variables, shrinkage of predictor weights, and model performance.
Results: we identified 12 prediction modelling studies on more than 1500 mass gatherings. Predictors of PPR and/or TTHR were, among others: accommodation (e.g. indoor versus outdoor), type of event (e.g. music concerts), and weather conditions. Nine out of 12 studies were prediction model development studies without validation; the other three were external validation studies of existing models. CHARMS criteria that were satisfied least often were reporting (in 17% of included studies) and subsequent handling of missing data (25%), and two criteria on model development (method of predictor variable selection, 22%, and shrinkage of predictor weights, 0%; Figure 1). Due to the retrospective nature of the studies, assessment of outcome blinded for predictors was trivial, whereas assessment of predictors blinded for outcome was impossible.
Conclusions: although the CHARMS checklist was developed for critical appraisal of clinical prediction modelling studies, it was useful for assessing risk of bias in prediction modelling studies on medical usage at mass gatherings as well. As such, it can be implemented in the GRADE evaluation of the body of evidence.
Patient or healthcare consumer involvement: this project, including the present systematic review and planned prediction model development, is conducted in close collaboration with the Relief Service at the Belgian Red Cross, which co-ordinates preventive aid campaigns at mass gatherings. Through regular meetings with central co-ordinators and representatives of local volunteers, we identified strengths and weaknesses of the current databases (pre-event application forms and post-event medical usage forms) and agreed about the desired features and predictive value of our own prediction model for medical usage rate at mass gatherings.
Objectives: to assess risk of bias of studies that developed and/or validated a regression model predicting patient presentation rate (PPR) or transfer to hospital rate (TTHR) at mass gatherings, using the CHARMS checklist.
Methods: we systematically searched for prediction modelling studies in six databases, classified included studies and extracted predictors for PPR or TTHR from multivariate regression models, based on the seven key issues pointed out in the first part of the CHARMS checklist. Next, we estimated risk of bias in 12 items, according to the second part of the CHARMS checklist. These items include general issues such as blinding of assessment and handling of missing data, but also specific issues for prediction modelling studies, such as method for selection of predictor variables, shrinkage of predictor weights, and model performance.
Results: we identified 12 prediction modelling studies on more than 1500 mass gatherings. Predictors of PPR and/or TTHR were, among others: accommodation (e.g. indoor versus outdoor), type of event (e.g. music concerts), and weather conditions. Nine out of 12 studies were prediction model development studies without validation; the other three were external validation studies of existing models. CHARMS criteria that were satisfied least often were reporting (in 17% of included studies) and subsequent handling of missing data (25%), and two criteria on model development (method of predictor variable selection, 22%, and shrinkage of predictor weights, 0%; Figure 1). Due to the retrospective nature of the studies, assessment of outcome blinded for predictors was trivial, whereas assessment of predictors blinded for outcome was impossible.
Conclusions: although the CHARMS checklist was developed for critical appraisal of clinical prediction modelling studies, it was useful for assessing risk of bias in prediction modelling studies on medical usage at mass gatherings as well. As such, it can be implemented in the GRADE evaluation of the body of evidence.
Patient or healthcare consumer involvement: this project, including the present systematic review and planned prediction model development, is conducted in close collaboration with the Relief Service at the Belgian Red Cross, which co-ordinates preventive aid campaigns at mass gatherings. Through regular meetings with central co-ordinators and representatives of local volunteers, we identified strengths and weaknesses of the current databases (pre-event application forms and post-event medical usage forms) and agreed about the desired features and predictive value of our own prediction model for medical usage rate at mass gatherings.