Quality Assessment and Evaluation Criteria in Supervised Learning

Amichai Painsky*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

3 Scopus citations

Abstract

Evaluating the performance of a learning algorithm is one of the basic tasks in machine learning and data science. In this chapter, we review commonly used performance measures and discuss their properties. We show that different measures focus on different aspects of the algorithm. Therefore, a learning algorithm is typically evaluated with respect to several criteria. We introduce conceptual tools and provide important guidelines for quality assessment of fully trained algorithms. We focus our attention to classification problems, as we draw connections to basic concepts in statistics, engineering, and other disciplines. In addition, we discuss regression problems, as we study popular residual-based measures. Finally, we suggest that evaluation criteria shall also be considered during the design of the algorithm. In this sense, the desired criteria determine the objective function, prior to the training of the algorithm. These design considerations are discussed, and several approaches are introduced to the problem.

Original languageEnglish
Title of host publicationMachine Learning for Data Science Handbook
Subtitle of host publicationData Mining and Knowledge Discovery Handbook, Third Edition
PublisherSpringer International Publishing
Pages171-195
Number of pages25
ISBN (Electronic)9783031246289
ISBN (Print)9783031246272
DOIs
StatePublished - 1 Jan 2023

Fingerprint

Dive into the research topics of 'Quality Assessment and Evaluation Criteria in Supervised Learning'. Together they form a unique fingerprint.

Cite this