Optimal universal learning and prediction of probabilistic concepts

A setup of the supervised learning problem is considered. Through this a universal predictor is sought. A classical result in universal coding (Gallager, 1976) states that the encoder attains the min-max redundancy. Recently, Merhav et al (1995) have shown that the performance of this predictor is a lower bound on the performance of any universal coder. In this paper, the proposed solution for the supervised learning problem is Bayesian, allowing the determination of an optimal way to choose the Bayesian 'prior' for the supervised learning problem, and observing the strong sequential, non-anticipating, structure of the resulting predictor.