Sequential Algorithms for Parameter Estimation Based on the Kullback-Leibler Information Measure

Methods of stochastic approximation are used to convert iterative algorithms for maximizing the Kullback-Leibler information measure into sequential algorithms. Special attention is given to the case of incomplete data, and several algorithms are presented to deal with situations of this kind. The application of these algorithms to the identification of finite impulse response (FIR) systems is considered. Issues such as convergence properties of the proposed algorithms, choice of initial conditions, the limit distribution, and the associated regularity conditions are beyond the scope of this correspondence. However, the existing literature on stochastic approximation, together with the ideas presented in this correspondence should provide the starting point for such analyses.