Clustering algorithms optimizer: A framework for large datasets

Roy Varshavsky, David Horn, Michal Linial

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Clustering algorithms are employed in many bioinformatics tasks, including categorization of protein sequences and analysis of gene-expression data. Although these algorithms are routinely applied, many of them suffer from the following limitations: (i) relying on predetermined parameters tuning, such as a-priori knowledge regarding the number of clusters; (ii) involving nondeterministic procedures that yield inconsistent outcomes. Thus, a framework that addresses these shortcomings is desirable. We provide a datadriven framework that includes two interrelated steps. The first one is SVDbased dimension reduction and the second is an automated tuning of the algorithm's parameter(s). The dimension reduction step is efficiently adjusted for very large datasets. The optimal parameter setting is identified according to the internal evaluation criterion known as Bayesian Information Criterion (BIC). This framework can incorporate most clustering algorithms and improve their performance. In this study we illustrate the effectiveness of this platform by incorporating the standard K-Means and the Quantum Clustering algorithms. The implementations are applied to several gene-expression benchmarks with significant success.

Original languageEnglish
Title of host publicationBioinformatics Research and Applications - Third International Symposium, ISBRA 2007, Proceedings
PublisherSpringer Verlag
Pages85-96
Number of pages12
ISBN (Print)3540720308, 9783540720300
DOIs
StatePublished - 2007
Event3rd International Symposium Bioinformatics Research and Applications, ISBRA 2007 - Atlanta, GA, United States
Duration: 7 May 200710 May 2007

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4463 LNBI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference3rd International Symposium Bioinformatics Research and Applications, ISBRA 2007
Country/TerritoryUnited States
CityAtlanta, GA
Period7/05/0710/05/07

Keywords

  • Bayesian Information Criterion (BIC)
  • Optimal K-Means (OKM)
  • Optimal Quantum Clustering (OQC)
  • Principal Component Analysis (PCA)
  • Quantum Clustering (QC)
  • Singular Value Decomposition (SVD)

Fingerprint

Dive into the research topics of 'Clustering algorithms optimizer: A framework for large datasets'. Together they form a unique fingerprint.

Cite this