Efficient Probability Mass Function Estimation from Partially Observed Data

Estimating the joint probability mass function (PMF) of a set of random variables from partially observed data is a crucial part of statistical learning and data analysis, with applications in areas such as recommender systems and data classification. Recently, it has been proposed to estimate the joint PMF based on the maximum likelihood (ML) of the data, fitted to a low-rank canonical polyadic decomposition (CPD) model of the joint PMF. To this end, a hybrid alternating-directions expectation-maximization (AD-EM) algorithm was proposed to solve the ML optimization problem, consisting of computationally expensive AD iterations followed by an EM refinement stage. It is well known that the convergence rate of EM decreases as the fraction of missing data increases. In this paper, we address the slow convergence of the EM algorithm. By adapting the squared iterative methods (SQUAREM) acceleration scheme to the context of PMF estimation, we propose the SQUAREM-PMF algorithm to speed up the convergence of the EM algorithm. Moreover, we demonstrate that running the computationally cheaper EM algorithm alone after an appropriate initialization is sufficient. Numerical results on both synthetic and real data in the context of movie recommendation show that our algorithm outperforms state-of-the-art PMF estimation algorithms.