A discriminative training algorithm for hidden Markov models

Assaf Ben-Yishai*, David Burshtein

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


We introduce a discriminative training algorithm for the estimation of hidden Markov model (HMM) parameters. This algorithm is based on an approximation of the maximum mutual information (MMI) objective function and its maximization in a technique similar to the expectation-maximization (EM) algorithm. The algorithm is implemented by a simple modification of the standard Baum-Welch algorithm, and can be applied to speech recognition as well as to word-spotting systems. Three tasks were tested: Isolated digit recognition in a noisy environment, connected digit recognition in a noisy environment and word-spotting. In all tasks a significant improvement over maximum likelihood (ML) estimation was observed. We also compared the new algorithm to the commonly used extended Baum-Welch MMI algorithm. In our tests the algorithm showed advantages in terms of both performance and computational complexity.

Original languageEnglish
Pages (from-to)204-217
Number of pages14
JournalIEEE Transactions on Speech and Audio Processing
Issue number3
StatePublished - May 2004


FundersFunder number
Ministry of Economy, Trade and Industry


    • Discriminative training
    • Hidden Markov model (HMM)
    • Maximum mutual information (MMI) criterion


    Dive into the research topics of 'A discriminative training algorithm for hidden Markov models'. Together they form a unique fingerprint.

    Cite this