Universal coding for arbitrarily varying sources and for hierarchies of model classes

Meir Feder*, Neri Merhav

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review


The minimum redundancy attainable by universal lossless codes for finite-state arbitrarily varying sources (AVS), and, in general, for an hierarchy of model classes, is investigated. In the AVS case, if the space of all possible underlying state sequences is partitioned into types, then the minimum universal coding redundancy can be essentially lower hounded by a quantity that decomposes into two terms, the first of which is the minimum redundancy within the type class (i.e., intra-type class redundancy), and the second is the minimum redundancy associated with a class of sources that can be thought of as `representatives' of the different types (i.e., inter-type class redundancy). This behavior can be generalized to universal coding for hierarchy of model classes, where each model class in the hierarchy has an increasing complexity. The lower bound for coding with respect to hierarchy of models is achievable by a Shannon code w.r.t an appropriate two-stage mixture, where the first stage mixture is over the sources in each class, and the second is a mixture over the indices of the model classes.

Original languageEnglish
Pages (from-to)82-91
Number of pages10
JournalProceedings of the Data Compression Conference
StatePublished - 1995
EventProceedings of the 5th Data Compression Conference - Snowbird, UT, USA
Duration: 28 Mar 199530 Mar 1995


Dive into the research topics of 'Universal coding for arbitrarily varying sources and for hierarchies of model classes'. Together they form a unique fingerprint.

Cite this