Optimality and stability of the K-hyperline clustering algorithm

Jayaraman J. Thiagarajan, Karthikeyan N. Ramamurthy, Andreas Spanias

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

K-hyperline clustering is an iterative algorithm based on singular value decomposition and it has been successfully used in sparse component analysis. In this paper, we prove that the algorithm converges to a locally optimal solution for a given set of training data, based on Lloyd's optimality conditions. Furthermore, the local optimality is shown by developing an Expectation- Maximization procedure for learning dictionaries to be used in sparse representations and by deriving the clustering algorithm as its special case. The cluster centroids obtained from the algorithm are proved to tessellate the space into convex Voronoi regions. The stability of clustering is shown by posing the problem as an empirical risk minimization procedure over a function class. It is proved that, under certain conditions, the cluster centroids learned from two sets of i.i.d. training samples drawn from the same probability space become arbitrarily close to each other, as the number of training samples increase asymptotically.

Original languageEnglish (US)
Pages (from-to)1299-1304
Number of pages6
JournalPattern Recognition Letters
Volume32
Issue number9
DOIs
StatePublished - Jul 1 2011

Fingerprint

Clustering algorithms
Singular value decomposition
Glossaries

Keywords

  • Empirical risk minimization
  • K-hyperline clustering
  • Optimality
  • Stability
  • Voronoi

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Signal Processing

Cite this

Optimality and stability of the K-hyperline clustering algorithm. / Thiagarajan, Jayaraman J.; Ramamurthy, Karthikeyan N.; Spanias, Andreas.

In: Pattern Recognition Letters, Vol. 32, No. 9, 01.07.2011, p. 1299-1304.

Research output: Contribution to journalArticle

Thiagarajan, Jayaraman J. ; Ramamurthy, Karthikeyan N. ; Spanias, Andreas. / Optimality and stability of the K-hyperline clustering algorithm. In: Pattern Recognition Letters. 2011 ; Vol. 32, No. 9. pp. 1299-1304.
@article{1af48b7ad51c4700a3bef34774f4502a,
title = "Optimality and stability of the K-hyperline clustering algorithm",
abstract = "K-hyperline clustering is an iterative algorithm based on singular value decomposition and it has been successfully used in sparse component analysis. In this paper, we prove that the algorithm converges to a locally optimal solution for a given set of training data, based on Lloyd's optimality conditions. Furthermore, the local optimality is shown by developing an Expectation- Maximization procedure for learning dictionaries to be used in sparse representations and by deriving the clustering algorithm as its special case. The cluster centroids obtained from the algorithm are proved to tessellate the space into convex Voronoi regions. The stability of clustering is shown by posing the problem as an empirical risk minimization procedure over a function class. It is proved that, under certain conditions, the cluster centroids learned from two sets of i.i.d. training samples drawn from the same probability space become arbitrarily close to each other, as the number of training samples increase asymptotically.",
keywords = "Empirical risk minimization, K-hyperline clustering, Optimality, Stability, Voronoi",
author = "Thiagarajan, {Jayaraman J.} and Ramamurthy, {Karthikeyan N.} and Andreas Spanias",
year = "2011",
month = "7",
day = "1",
doi = "10.1016/j.patrec.2011.03.005",
language = "English (US)",
volume = "32",
pages = "1299--1304",
journal = "Pattern Recognition Letters",
issn = "0167-8655",
publisher = "Elsevier",
number = "9",

}

TY - JOUR

T1 - Optimality and stability of the K-hyperline clustering algorithm

AU - Thiagarajan, Jayaraman J.

AU - Ramamurthy, Karthikeyan N.

AU - Spanias, Andreas

PY - 2011/7/1

Y1 - 2011/7/1

N2 - K-hyperline clustering is an iterative algorithm based on singular value decomposition and it has been successfully used in sparse component analysis. In this paper, we prove that the algorithm converges to a locally optimal solution for a given set of training data, based on Lloyd's optimality conditions. Furthermore, the local optimality is shown by developing an Expectation- Maximization procedure for learning dictionaries to be used in sparse representations and by deriving the clustering algorithm as its special case. The cluster centroids obtained from the algorithm are proved to tessellate the space into convex Voronoi regions. The stability of clustering is shown by posing the problem as an empirical risk minimization procedure over a function class. It is proved that, under certain conditions, the cluster centroids learned from two sets of i.i.d. training samples drawn from the same probability space become arbitrarily close to each other, as the number of training samples increase asymptotically.

AB - K-hyperline clustering is an iterative algorithm based on singular value decomposition and it has been successfully used in sparse component analysis. In this paper, we prove that the algorithm converges to a locally optimal solution for a given set of training data, based on Lloyd's optimality conditions. Furthermore, the local optimality is shown by developing an Expectation- Maximization procedure for learning dictionaries to be used in sparse representations and by deriving the clustering algorithm as its special case. The cluster centroids obtained from the algorithm are proved to tessellate the space into convex Voronoi regions. The stability of clustering is shown by posing the problem as an empirical risk minimization procedure over a function class. It is proved that, under certain conditions, the cluster centroids learned from two sets of i.i.d. training samples drawn from the same probability space become arbitrarily close to each other, as the number of training samples increase asymptotically.

KW - Empirical risk minimization

KW - K-hyperline clustering

KW - Optimality

KW - Stability

KW - Voronoi

UR - http://www.scopus.com/inward/record.url?scp=79953864838&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79953864838&partnerID=8YFLogxK

U2 - 10.1016/j.patrec.2011.03.005

DO - 10.1016/j.patrec.2011.03.005

M3 - Article

AN - SCOPUS:79953864838

VL - 32

SP - 1299

EP - 1304

JO - Pattern Recognition Letters

JF - Pattern Recognition Letters

SN - 0167-8655

IS - 9

ER -