TY - JOUR
T1 - The uniform distribution as a universal prior
AU - Shulman, Nadav
AU - Feder, Meir
PY - 2004/6
Y1 - 2004/6
N2 - In this correspondence, we discuss the properties of the uniform prior as a universal prior, i.e., a prior that induces a mutual information that is simultaneously close to the capacity for all channels. We determine bounds on the amount of the mutual information loss in using the uniform prior instead of the capacity-achieving prior. Specifically, for the class of binary input channels with any output alphabet, we show that the Z-channel has the minimal mutual information with uniform prior, out of all channels with a given capacity. From this, we conclude that the degradation of the mutual information with respect to the capacity is at most 0.011 bit, and as was shown previously, at most 6%. A related result is that the capacity-achieving prior, for any channel, is not far from uniform. Some of these results are extended to channels with nonbinary input.
AB - In this correspondence, we discuss the properties of the uniform prior as a universal prior, i.e., a prior that induces a mutual information that is simultaneously close to the capacity for all channels. We determine bounds on the amount of the mutual information loss in using the uniform prior instead of the capacity-achieving prior. Specifically, for the class of binary input channels with any output alphabet, we show that the Z-channel has the minimal mutual information with uniform prior, out of all channels with a given capacity. From this, we conclude that the degradation of the mutual information with respect to the capacity is at most 0.011 bit, and as was shown previously, at most 6%. A related result is that the capacity-achieving prior, for any channel, is not far from uniform. Some of these results are extended to channels with nonbinary input.
KW - Uniform distribution
KW - Universal prior
KW - Z-channel
UR - http://www.scopus.com/inward/record.url?scp=2942655233&partnerID=8YFLogxK
U2 - 10.1109/TIT.2004.828152
DO - 10.1109/TIT.2004.828152
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.letter???
AN - SCOPUS:2942655233
SN - 0018-9448
VL - 50
SP - 1356
EP - 1362
JO - IEEE Transactions on Information Theory
JF - IEEE Transactions on Information Theory
IS - 6
ER -