Publication

Graphical assessment of incremental value of novel markers in prediction models: From statistical to decision analytical perspectives

Steyerberg, E. W., Vedder, M. M., Leening, M. J. G., Postmus, D., D'Agostino, R. B., Van Calster, B. & Pencina, M. J., Jul-2015, In : BIOMETRICAL JOURNAL. 57, 4, p. 556-570 15 p.

Research output: Contribution to journalArticleAcademicpeer-review

Copy link to clipboard

Documents

  • Graphical assessment of incremental value of novel markers in prediction

    Final publisher's version, 619 KB, PDF document

    Request copy

DOI

  • Ewout W. Steyerberg
  • Moniek M. Vedder
  • Maarten J. G. Leening
  • Douwe Postmus
  • Ralph B. D'Agostino
  • Ben Van Calster
  • Michael J. Pencina

New markers may improve prediction of diagnostic and prognostic outcomes. We aimed to review options for graphical display and summary measures to assess the predictive value of markers over standard, readily available predictors. We illustrated various approaches using previously published data on 3264 participants from the Framingham Heart Study, where 183 developed coronary heart disease (10-year risk 5.6%). We considered performance measures for the incremental value of adding HDL cholesterol to a prediction model. An initial assessment may consider statistical significance (HR = 0.65, 95% confidence interval 0.53 to 0.80; likelihood ratio p <0.001), and distributions of predicted risks (densities or box plots) with various summary measures. A range of decision thresholds is considered in predictiveness and receiver operating characteristic curves, where the area under the curve (AUC) increased from 0.762 to 0.774 by adding HDL. We can furthermore focus on reclassification of participants with and without an event in a reclassification graph, with the continuous net reclassification improvement (NRI) as a summary measure. When we focus on one particular decision threshold, the changes in sensitivity and specificity are central. We propose a net reclassification risk graph, which allows us to focus on the number of reclassified persons and their event rates. Summary measures include the binary AUC, the two-category NRI, and decision analytic variants such as the net benefit (NB). Various graphs and summary measures can be used to assess the incremental predictive value of a marker. Important insights for impact on decision making are provided by a simple graph for the net reclassification risk.

Original languageEnglish
Pages (from-to)556-570
Number of pages15
JournalBIOMETRICAL JOURNAL
Volume57
Issue number4
Publication statusPublished - Jul-2015

    Keywords

  • Decision analysis, Reclassification, Regression analysis, Risk assessment, ROC curve, LOGISTIC-REGRESSION MODELS, RELATIVE UTILITY CURVES, RISK PREDICTION, NET RECLASSIFICATION, CARDIOVASCULAR RISK, ROC CURVE, DISCRIMINATION, COEFFICIENT, PERFORMANCE, IMPROVEMENT

ID: 23507972