TY - JOUR
T1 - Maximum Entropy as a Special Case of the Minimum Description Length Criterion
AU - Feder, Meir
PY - 1986/11
Y1 - 1986/11
N2 - The Maximum Entropy (ME) and Maximum Likelihood (ML) criteria are the bases for two approaches to statistical inference problems. A new criterion, called the Minimum Description Length (MDL), has been recently introduced. This criterion generalizes the ML method so it can be applied to more general situations, e.g., when the number of parameters is unknown. It is shown that ME is also a special case of the MDL criterion; maximizing the entropy subject to some constraints on the underlying probability function is identical to minimizing the code length required to represent all possible i.i.d. realizations of the random variable such that the sample frequencies (or histogram) satisfy those given constraints.
AB - The Maximum Entropy (ME) and Maximum Likelihood (ML) criteria are the bases for two approaches to statistical inference problems. A new criterion, called the Minimum Description Length (MDL), has been recently introduced. This criterion generalizes the ML method so it can be applied to more general situations, e.g., when the number of parameters is unknown. It is shown that ME is also a special case of the MDL criterion; maximizing the entropy subject to some constraints on the underlying probability function is identical to minimizing the code length required to represent all possible i.i.d. realizations of the random variable such that the sample frequencies (or histogram) satisfy those given constraints.
UR - http://www.scopus.com/inward/record.url?scp=0022807153&partnerID=8YFLogxK
U2 - 10.1109/TIT.1986.1057237
DO - 10.1109/TIT.1986.1057237
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
AN - SCOPUS:0022807153
VL - 32
SP - 847
EP - 849
JO - IEEE Transactions on Information Theory
JF - IEEE Transactions on Information Theory
SN - 0018-9448
IS - 6
ER -