DAVID G. BROWN

The course will cover the broad range of performance metrics that go beyond the simple calculation of success rate or accuracy on a limited test set. These include sensitivity, specificity, positive and negative predictive value, receiver operating characteristic (ROC) curve, and area under the ROC curve (AUC). Particular attention will be paid to the ROC curve formalism, and a hands-on demonstration will be given. Participants will be asked to bring a laptop or other computational device and will take part in simulated reader studies to better understand ROC methodology and to compete for prizes. Both parametric and nonparametric uncertainty calculation techniques will be discussed, and the importance of good data hygiene in maintaining complete separation of training and test data will be illustrated.