TY - GEN

T1 - Relations between entropy and error probability

AU - Feder, Meir

AU - Merhav, Neri

PY - 1993

Y1 - 1993

N2 - The relation between the entropy of a discrete random variable and the minimum attainable probability of error made in guessing its value is examined. While Fano's inequality provides a tight lower bound on the error probability in terms of the entropy, we derive a converse result - a tight upper bound on the minimal error probability in terms of the entropy. As a consequence of this relation, a channel coding theorem for the equivocation is presented. At a rate R < C, where C is the channel capacity, it follows straightforwardly from the classical channel coding theorem and the bounds above that the equivocation can be made arbitrarily small (exponentially fast with the block length). This result is proved directly for DMC's, and from this proof it is further concluded that for R ≥ C the equivocation achieve its minimal value of R - C at the rate of n- 1/2 , where n is the block length.

AB - The relation between the entropy of a discrete random variable and the minimum attainable probability of error made in guessing its value is examined. While Fano's inequality provides a tight lower bound on the error probability in terms of the entropy, we derive a converse result - a tight upper bound on the minimal error probability in terms of the entropy. As a consequence of this relation, a channel coding theorem for the equivocation is presented. At a rate R < C, where C is the channel capacity, it follows straightforwardly from the classical channel coding theorem and the bounds above that the equivocation can be made arbitrarily small (exponentially fast with the block length). This result is proved directly for DMC's, and from this proof it is further concluded that for R ≥ C the equivocation achieve its minimal value of R - C at the rate of n- 1/2 , where n is the block length.

UR - http://www.scopus.com/inward/record.url?scp=0027311707&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:0027311707

SN - 0780308786

T3 - Proceedings of the 1993 IEEE International Symposium on Information Theory

SP - 72

BT - Proceedings of the 1993 IEEE International Symposium on Information Theory

PB - Publ by IEEE

T2 - Proceedings of the 1993 IEEE International Symposium on Information Theory

Y2 - 17 January 1993 through 22 January 1993

ER -