TY - GEN

T1 - Infinite Gaussian Mixture Modeling with an Improved Estimation of the Number of Clusters

AU - Matza, Avi

AU - Bistritz, Yuval

N1 - Publisher Copyright:
© 2021, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

PY - 2021

Y1 - 2021

N2 - Infinite Gaussian mixture modeling (IGMM) is a modeling method that determines all the parameters of a Gaussian mixture model (GMM), including its order. It has been well documented that it is a consistent estimator for probability density functions in the sense that, given enough training data from sufficiently regular probability density functions, it will converge to the shape of the original density curve. It is also known, however, that IGMM provides an inconsistent estimation of the number of clusters. The current paper shows that the nature of this inconsistency is an overestimation, and we pinpoint that this problem is an inherent part of the training algorithm. It stems mostly from a “self-reinforcing feedback” which is a certain relation between the likelihood function of one of the model hyperparameters (α) and the probability of sampling the number of components, that sustain their mutual growth during the Gibbs iterations. We show that this problem can be resolved by using informative priors for α and propose a modified training procedure that uses the inverse χ2 for this purpose. The modified algorithm successfully recovers the “known” order in all the experiments with synthetic data sets. It also demonstrates good results when compared to other methods used to evaluate model order, using real-world databases. Furthermore, the improved performance is attained without undermining the fidelity of estimating the original PDFs and with a significant reduction in computational cost.

AB - Infinite Gaussian mixture modeling (IGMM) is a modeling method that determines all the parameters of a Gaussian mixture model (GMM), including its order. It has been well documented that it is a consistent estimator for probability density functions in the sense that, given enough training data from sufficiently regular probability density functions, it will converge to the shape of the original density curve. It is also known, however, that IGMM provides an inconsistent estimation of the number of clusters. The current paper shows that the nature of this inconsistency is an overestimation, and we pinpoint that this problem is an inherent part of the training algorithm. It stems mostly from a “self-reinforcing feedback” which is a certain relation between the likelihood function of one of the model hyperparameters (α) and the probability of sampling the number of components, that sustain their mutual growth during the Gibbs iterations. We show that this problem can be resolved by using informative priors for α and propose a modified training procedure that uses the inverse χ2 for this purpose. The modified algorithm successfully recovers the “known” order in all the experiments with synthetic data sets. It also demonstrates good results when compared to other methods used to evaluate model order, using real-world databases. Furthermore, the improved performance is attained without undermining the fidelity of estimating the original PDFs and with a significant reduction in computational cost.

UR - http://www.scopus.com/inward/record.url?scp=85130094961&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85130094961

T3 - 35th AAAI Conference on Artificial Intelligence, AAAI 2021

SP - 8921

EP - 8929

BT - 35th AAAI Conference on Artificial Intelligence, AAAI 2021

PB - Association for the Advancement of Artificial Intelligence

Y2 - 2 February 2021 through 9 February 2021

ER -