Article type
Year
Abstract
Background: as part of the movement towards personalised medicine, the literature on prognostic model studies is increasing. Numerous models are developed for the same disease and outcome. A clinician may not be able to identify all relevant models and assess whether they are suitable for use in clinical practice and applicable to their patients. To decide this, a systematic summary of available models and their performance in external cohorts is necessary. Our project is the first exemplar Cochrane Review for prognostic models concerning previously untreated patients with chronic lymphocytic leukaemia (CLL), and aims to contribute to future high-quality Cochrane Reviews in any clinical area.
Objectives: this systematic review aims to identify and assess all prognostic models developed in untreated patients with CLL, and their external validation studies. We focus on the particular challenges encountered when synthesizing prognostic model studies.
Methods: we systematically searched MEDLINE via OvidSP (September 2018) for primary studies that developed or externally validated prognostic models to predict overall survival, time-to-first-treatment or progression-free survival. Two review authors independently screened the publications for eligibility, extracted the relevant characteristics and performance measures and assessed the methodological quality of the studies with the Prediction model Risk Of Bias ASsessment Tool (PROBAST). When sufficient data for a model were available, we pooled the measures for calibration and discrimination. (Funding: BMBF, 01KG1711.)
Results: due to the unavailability of specific search filters, our sensitive search strategy yielded 16,047 references. The demanding screening process was complicated by the blurred line between prognostic models and similar concepts such as staging systems. To cover the broad range of publication formats, study designs and performance measures, we developed a data extraction form over several pilot rounds. We observed a paucity of external validation studies, with only nine prognostic models (out of 43 eligible models) undergoing at least one external validation. We observed reporting deficiencies in both development and validation studies. Consequently, quality and applicability assessment with PROBAST was challenging due to lack of information and the open nature of the signalling questions. At each review stage, we resolved differences by group discussion.
Conclusions: the methodology underpinning prognostic model reviews in Cochrane is still evolving. Currently, lack of specific search strategies and severe reporting deficiencies in the prognostic model literature impede the process.
Patient or healthcare consumer involvement: patients will benefit as the efficacy and quality of all available CLL models will be summarized, aiding their personalised approach to treatment. The project will also serve as an exemplar for prognostic model reviews in other clinical areas
Objectives: this systematic review aims to identify and assess all prognostic models developed in untreated patients with CLL, and their external validation studies. We focus on the particular challenges encountered when synthesizing prognostic model studies.
Methods: we systematically searched MEDLINE via OvidSP (September 2018) for primary studies that developed or externally validated prognostic models to predict overall survival, time-to-first-treatment or progression-free survival. Two review authors independently screened the publications for eligibility, extracted the relevant characteristics and performance measures and assessed the methodological quality of the studies with the Prediction model Risk Of Bias ASsessment Tool (PROBAST). When sufficient data for a model were available, we pooled the measures for calibration and discrimination. (Funding: BMBF, 01KG1711.)
Results: due to the unavailability of specific search filters, our sensitive search strategy yielded 16,047 references. The demanding screening process was complicated by the blurred line between prognostic models and similar concepts such as staging systems. To cover the broad range of publication formats, study designs and performance measures, we developed a data extraction form over several pilot rounds. We observed a paucity of external validation studies, with only nine prognostic models (out of 43 eligible models) undergoing at least one external validation. We observed reporting deficiencies in both development and validation studies. Consequently, quality and applicability assessment with PROBAST was challenging due to lack of information and the open nature of the signalling questions. At each review stage, we resolved differences by group discussion.
Conclusions: the methodology underpinning prognostic model reviews in Cochrane is still evolving. Currently, lack of specific search strategies and severe reporting deficiencies in the prognostic model literature impede the process.
Patient or healthcare consumer involvement: patients will benefit as the efficacy and quality of all available CLL models will be summarized, aiding their personalised approach to treatment. The project will also serve as an exemplar for prognostic model reviews in other clinical areas