TY - JOUR
T1 - Deep Individual Active Learning
T2 - Safeguarding against Out-of-Distribution Challenges in Neural Networks
AU - Shayovitz, Shachar
AU - Bibas, Koby
AU - Feder, Meir
N1 - Publisher Copyright:
© 2024 by the authors.
PY - 2024/2
Y1 - 2024/2
N2 - Active learning (AL) is a paradigm focused on purposefully selecting training data to enhance a model’s performance by minimizing the need for annotated samples. Typically, strategies assume that the training pool shares the same distribution as the test set, which is not always valid in privacy-sensitive applications where annotating user data is challenging. In this study, we operate within an individual setting and leverage an active learning criterion which selects data points for labeling based on minimizing the min-max regret on a small unlabeled test set sample. Our key contribution lies in the development of an efficient algorithm, addressing the challenging computational complexity associated with approximating this criterion for neural networks. Notably, our results show that, especially in the presence of out-of-distribution data, the proposed algorithm substantially reduces the required training set size by up to 15.4%, 11%, and 35.1% for CIFAR10, EMNIST, and MNIST datasets, respectively.
AB - Active learning (AL) is a paradigm focused on purposefully selecting training data to enhance a model’s performance by minimizing the need for annotated samples. Typically, strategies assume that the training pool shares the same distribution as the test set, which is not always valid in privacy-sensitive applications where annotating user data is challenging. In this study, we operate within an individual setting and leverage an active learning criterion which selects data points for labeling based on minimizing the min-max regret on a small unlabeled test set sample. Our key contribution lies in the development of an efficient algorithm, addressing the challenging computational complexity associated with approximating this criterion for neural networks. Notably, our results show that, especially in the presence of out-of-distribution data, the proposed algorithm substantially reduces the required training set size by up to 15.4%, 11%, and 35.1% for CIFAR10, EMNIST, and MNIST datasets, respectively.
KW - active learning
KW - deep active learning
KW - individual sequences
KW - normalized maximum likelihood
KW - out-of-distribution
KW - universal prediction
UR - http://www.scopus.com/inward/record.url?scp=85187301318&partnerID=8YFLogxK
U2 - 10.3390/e26020129
DO - 10.3390/e26020129
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
C2 - 38392384
AN - SCOPUS:85187301318
SN - 1099-4300
VL - 26
JO - Entropy
JF - Entropy
IS - 2
M1 - 129
ER -