Abstract
Evaluating the performance of a learning algorithm is one of the basic tasks in machine learning and data science. In this chapter, we review commonly used performance measures and discuss their properties. We show that different measures focus on different aspects of the algorithm. Therefore, a learning algorithm is typically evaluated with respect to several criteria. We introduce conceptual tools and provide important guidelines for quality assessment of fully trained algorithms. We focus our attention to classification problems, as we draw connections to basic concepts in statistics, engineering, and other disciplines. In addition, we discuss regression problems, as we study popular residual-based measures. Finally, we suggest that evaluation criteria shall also be considered during the design of the algorithm. In this sense, the desired criteria determine the objective function, prior to the training of the algorithm. These design considerations are discussed, and several approaches are introduced to the problem.
Original language | English |
---|---|
Title of host publication | Machine Learning for Data Science Handbook |
Subtitle of host publication | Data Mining and Knowledge Discovery Handbook, Third Edition |
Publisher | Springer International Publishing |
Pages | 171-195 |
Number of pages | 25 |
ISBN (Electronic) | 9783031246289 |
ISBN (Print) | 9783031246272 |
DOIs | |
State | Published - 1 Jan 2023 |