Article type
Year
Abstract
Background: Meta-epidemiological studies have shown that study results are directly influenced by study design characteristics. The results of a randomized trial may for example be biased by inadequate allocation concealment and diagnostic test accuracy may be overestimated in case-control studies. The influence of design features on the results of prognostic research remains unclear.
Objectives: To determine which study characteristics influence performance of a prognostic model upon external validation, taking the validations of three established prediction models for cardiovascular disease (CVD) as an example.
Methods: In December 2015, MEDLINE, Embase, Web of Science, and Scopus were searched for articles investigating the external validation of three CVD risk equations (Framingham Wilson 1998, Framingham ATP III 2002 and Pooled Cohort Equations (PCE) 2013). Studies published before June 2013 were identified from a previous review. Studies were eligible if they validated the original prediction model in a general population setting. Data were extracted on key study characteristics. Random-effects meta-regression will be used to determine which study characteristics influence model performance (c-statistic and observed/expected ratio).
Results: The search identified 10,687 references, of which 1501 were screened in full text and 47 met our eligibility criteria. These articles described the external validation of Framingham Wilson (27 articles), Framingham ATP III (16 articles) and the PCE (10 articles). The c-statistic varied between 0.56 and 0.92. We will investigate sources of heterogeneity and present the range of performance for different design characteristics, including study design (e.g. cohort), sample size, assessment of outcomes, and handling of missing data.
Conclusions: This study will identify design characteristics influencing the performance of CVD risk prediction models in external validation studies, and thereby facilitate risk of bias assessment in systematic reviews of prognostic studies.
Objectives: To determine which study characteristics influence performance of a prognostic model upon external validation, taking the validations of three established prediction models for cardiovascular disease (CVD) as an example.
Methods: In December 2015, MEDLINE, Embase, Web of Science, and Scopus were searched for articles investigating the external validation of three CVD risk equations (Framingham Wilson 1998, Framingham ATP III 2002 and Pooled Cohort Equations (PCE) 2013). Studies published before June 2013 were identified from a previous review. Studies were eligible if they validated the original prediction model in a general population setting. Data were extracted on key study characteristics. Random-effects meta-regression will be used to determine which study characteristics influence model performance (c-statistic and observed/expected ratio).
Results: The search identified 10,687 references, of which 1501 were screened in full text and 47 met our eligibility criteria. These articles described the external validation of Framingham Wilson (27 articles), Framingham ATP III (16 articles) and the PCE (10 articles). The c-statistic varied between 0.56 and 0.92. We will investigate sources of heterogeneity and present the range of performance for different design characteristics, including study design (e.g. cohort), sample size, assessment of outcomes, and handling of missing data.
Conclusions: This study will identify design characteristics influencing the performance of CVD risk prediction models in external validation studies, and thereby facilitate risk of bias assessment in systematic reviews of prognostic studies.