Optimal finite state universal coding of individual sequences

Eado Meron*, Meir Feder

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

3 Scopus citations


The problem of assigning a probability to the next outcome of an individual binary sequence under the constraint that the universal predictor has a finite number of states, is explored. The two main loss functions that are considered are the square error loss and the self-information loss. Universal prediction w.r.t. the self-information loss can be combined with arithmetic encoding (see [9]) to construct a universal encoder, thus we essentially explore the universal coding problem. We analyze the performance of randomized time-invariant K-state universal predictors, and provide performance bounds in terms of the number of states K for long enough sequences. In the case where the comparison class consists of constant predictors we provide, for the square error loss, tight bounds indicating that the optimal asymptotic expected redundancy is O (1/K). For the self-information loss we show an upper bound on the coding redundancy of O (log K/K) and a lower bound of O(1/K).

Original languageEnglish
Pages (from-to)332-341
Number of pages10
JournalProceedings of the Data Compression Conference
StatePublished - 2004
EventProceedings - DCC 2004 Data Compression Conference - Snowbird, UT., United States
Duration: 23 Mar 200425 Mar 2004


Dive into the research topics of 'Optimal finite state universal coding of individual sequences'. Together they form a unique fingerprint.

Cite this