Permutation Invariant Individual Batch Learning

Yaniv Fogel*, Meir Feder

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


This paper considers the individual batch learning problem. Batch learning (in contrast to online) refers to the case where there is a "batch"of training data and the goal is to predict a test outcome. Individual learning refers to the case where the data (training and test) is arbitrary, individual. This batch individual setting poses a fundamental issue of defining a plausible criterion for a universal learner since in each experiment there is a single test sample. We propose a permutation invariant criterion that, intuitively, lets the individual training sequence manifest its empirical structure for predicting the test sample. This criterion is essentially a min-max regret, where the regret is based on a leave-one-out approach, minimized over the universal learner and maximized over the outcome sequences (thus agnostic). To show its plausibility, we analyze the criterion and its resulting learner for two cases: Binary Bernoulli and 1-D deterministic barrier. For both cases the regret behaves as O(c/N), N the size of the training and c = 1 for the Bernoulli case and log4 for the 1-D barrier. Interestingly, in the Bernoulli case, the regret in the stochastic setting behaves as O(1/2N) while here, in the individual setting, it has a larger constant.

Original languageEnglish
Title of host publication2023 IEEE Information Theory Workshop, ITW 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages5
ISBN (Electronic)9798350301496
StatePublished - 2023
Event2023 IEEE Information Theory Workshop, ITW 2023 - Saint-Malo, France
Duration: 23 Apr 202328 Apr 2023

Publication series

Name2023 IEEE Information Theory Workshop, ITW 2023


Conference2023 IEEE Information Theory Workshop, ITW 2023


Dive into the research topics of 'Permutation Invariant Individual Batch Learning'. Together they form a unique fingerprint.

Cite this