The uniform distribution as a universal prior

Nadav Shulman*, Meir Feder

*Corresponding author for this work

Research output: Contribution to journalLetterpeer-review

45 Scopus citations

Abstract

In this correspondence, we discuss the properties of the uniform prior as a universal prior, i.e., a prior that induces a mutual information that is simultaneously close to the capacity for all channels. We determine bounds on the amount of the mutual information loss in using the uniform prior instead of the capacity-achieving prior. Specifically, for the class of binary input channels with any output alphabet, we show that the Z-channel has the minimal mutual information with uniform prior, out of all channels with a given capacity. From this, we conclude that the degradation of the mutual information with respect to the capacity is at most 0.011 bit, and as was shown previously, at most 6%. A related result is that the capacity-achieving prior, for any channel, is not far from uniform. Some of these results are extended to channels with nonbinary input.

Original languageEnglish
Pages (from-to)1356-1362
Number of pages7
JournalIEEE Transactions on Information Theory
Volume50
Issue number6
DOIs
StatePublished - Jun 2004

Keywords

  • Uniform distribution
  • Universal prior
  • Z-channel

Fingerprint

Dive into the research topics of 'The uniform distribution as a universal prior'. Together they form a unique fingerprint.

Cite this