Coarse-grained localized diffusion

Guy Wolf, Aviv Rotbart, Gil David, Amir Averbuch

Research output: Contribution to journalArticlepeer-review

Abstract

Data-analysis methods nowadays are expected to deal with increasingly large amounts of data. Such massive datasets often contain many redundancies. One effect from these redundancies is the high dimensionality of datasets, which is handled by dimensionality reduction techniques. Another effect is the duplicity of very similar observations (or data-points) that can be analyzed together as a cluster. We propose an approach for dealing with both effects by coarse-graining the popular Diffusion Maps (DM) dimensionality reduction framework from the data-point level to the cluster level. This way, the size of the analyzed dataset is decreased by only referring to clusters instead of individual data-points. Then, the dimensionality of the dataset can be decreased by the DM embedding. We show that the essential properties (e.g., ergodicity) of the underlying diffusion process of DM are preserved by the coarse-graining. The affinity that is generated by the coarse-grained process, which we call Localized Diffusion Process (LDP), is strongly related to the recently introduced Localized Diffusion Folders (LDF) [G. David, A. Averbuch, Hierarchical data organization, clustering and denoising via localized diffusion folders, Appl. Comput. Harmon. Anal. (2011), in press] hierarchical clustering algorithm. We show that the LDP coarse-graining is in fact equivalent to the affinity-pruning that is achieved at each folder-level in the LDF hierarchy.

Original languageEnglish
Pages (from-to)388-400
Number of pages13
JournalApplied and Computational Harmonic Analysis
Volume33
Issue number3
DOIs
StatePublished - Nov 2012

Keywords

  • Coarse-graining
  • Diffusion maps
  • Dimensionality reduction
  • Localized diffusion folders

Fingerprint

Dive into the research topics of 'Coarse-grained localized diffusion'. Together they form a unique fingerprint.

Cite this