A new optimization criterion for generalized discriminant analysis on undersampled problems

Jieping Ye, Ravi Janardan, Cheong Hee Park, Haesun Park

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

A new optimization criterion for discriminant analysis is presented. The new criterion extends the optimization criteria of the classical linear discriminant analysis (LDA) by introducing the pseudo-inverse when the scatter matrices are singular. It is applicable regardless of the relative sizes of the data dimension and sample size, overcoming a limitation of the classical LDA. Recently, a new algorithm called LDA/GSVD for structure-preserving dimension reduction has been introduced, which extends the classical LDA to very high-dimensional undersampled problems by using the generalized singular value decomposition (GSVD). The solution from the LDA/GSVD algorithm is a special case of the solution for our generalized criterion in this paper, which is also based on GSVD. We also present an approximate solution for our GSVDbased solution, which reduces computational complexity by finding sub-clusters of each cluster, and using their centroids to capture the structure of each cluster. This reduced problem yields much smaller matrices of which the GSVD can be applied efficiently. Experiments on text data, with up to 7000 dimensions, show that the approximation algorithm produces results that are close to those produced by the exact algorithm.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE International Conference on Data Mining, ICDM
Pages419-426
Number of pages8
StatePublished - 2003
Externally publishedYes
Event3rd IEEE International Conference on Data Mining, ICDM '03 - Melbourne, FL, United States
Duration: Nov 19 2003Nov 22 2003

Other

Other3rd IEEE International Conference on Data Mining, ICDM '03
CountryUnited States
CityMelbourne, FL
Period11/19/0311/22/03

Fingerprint

Discriminant analysis
Singular value decomposition
Approximation algorithms
Computational complexity
Experiments

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Ye, J., Janardan, R., Park, C. H., & Park, H. (2003). A new optimization criterion for generalized discriminant analysis on undersampled problems. In Proceedings - IEEE International Conference on Data Mining, ICDM (pp. 419-426)

A new optimization criterion for generalized discriminant analysis on undersampled problems. / Ye, Jieping; Janardan, Ravi; Park, Cheong Hee; Park, Haesun.

Proceedings - IEEE International Conference on Data Mining, ICDM. 2003. p. 419-426.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ye, J, Janardan, R, Park, CH & Park, H 2003, A new optimization criterion for generalized discriminant analysis on undersampled problems. in Proceedings - IEEE International Conference on Data Mining, ICDM. pp. 419-426, 3rd IEEE International Conference on Data Mining, ICDM '03, Melbourne, FL, United States, 11/19/03.
Ye J, Janardan R, Park CH, Park H. A new optimization criterion for generalized discriminant analysis on undersampled problems. In Proceedings - IEEE International Conference on Data Mining, ICDM. 2003. p. 419-426
Ye, Jieping ; Janardan, Ravi ; Park, Cheong Hee ; Park, Haesun. / A new optimization criterion for generalized discriminant analysis on undersampled problems. Proceedings - IEEE International Conference on Data Mining, ICDM. 2003. pp. 419-426
@inproceedings{66a4eae1d4a842929cd61b84440fb3bb,
title = "A new optimization criterion for generalized discriminant analysis on undersampled problems",
abstract = "A new optimization criterion for discriminant analysis is presented. The new criterion extends the optimization criteria of the classical linear discriminant analysis (LDA) by introducing the pseudo-inverse when the scatter matrices are singular. It is applicable regardless of the relative sizes of the data dimension and sample size, overcoming a limitation of the classical LDA. Recently, a new algorithm called LDA/GSVD for structure-preserving dimension reduction has been introduced, which extends the classical LDA to very high-dimensional undersampled problems by using the generalized singular value decomposition (GSVD). The solution from the LDA/GSVD algorithm is a special case of the solution for our generalized criterion in this paper, which is also based on GSVD. We also present an approximate solution for our GSVDbased solution, which reduces computational complexity by finding sub-clusters of each cluster, and using their centroids to capture the structure of each cluster. This reduced problem yields much smaller matrices of which the GSVD can be applied efficiently. Experiments on text data, with up to 7000 dimensions, show that the approximation algorithm produces results that are close to those produced by the exact algorithm.",
author = "Jieping Ye and Ravi Janardan and Park, {Cheong Hee} and Haesun Park",
year = "2003",
language = "English (US)",
isbn = "0769519784",
pages = "419--426",
booktitle = "Proceedings - IEEE International Conference on Data Mining, ICDM",

}

TY - GEN

T1 - A new optimization criterion for generalized discriminant analysis on undersampled problems

AU - Ye, Jieping

AU - Janardan, Ravi

AU - Park, Cheong Hee

AU - Park, Haesun

PY - 2003

Y1 - 2003

N2 - A new optimization criterion for discriminant analysis is presented. The new criterion extends the optimization criteria of the classical linear discriminant analysis (LDA) by introducing the pseudo-inverse when the scatter matrices are singular. It is applicable regardless of the relative sizes of the data dimension and sample size, overcoming a limitation of the classical LDA. Recently, a new algorithm called LDA/GSVD for structure-preserving dimension reduction has been introduced, which extends the classical LDA to very high-dimensional undersampled problems by using the generalized singular value decomposition (GSVD). The solution from the LDA/GSVD algorithm is a special case of the solution for our generalized criterion in this paper, which is also based on GSVD. We also present an approximate solution for our GSVDbased solution, which reduces computational complexity by finding sub-clusters of each cluster, and using their centroids to capture the structure of each cluster. This reduced problem yields much smaller matrices of which the GSVD can be applied efficiently. Experiments on text data, with up to 7000 dimensions, show that the approximation algorithm produces results that are close to those produced by the exact algorithm.

AB - A new optimization criterion for discriminant analysis is presented. The new criterion extends the optimization criteria of the classical linear discriminant analysis (LDA) by introducing the pseudo-inverse when the scatter matrices are singular. It is applicable regardless of the relative sizes of the data dimension and sample size, overcoming a limitation of the classical LDA. Recently, a new algorithm called LDA/GSVD for structure-preserving dimension reduction has been introduced, which extends the classical LDA to very high-dimensional undersampled problems by using the generalized singular value decomposition (GSVD). The solution from the LDA/GSVD algorithm is a special case of the solution for our generalized criterion in this paper, which is also based on GSVD. We also present an approximate solution for our GSVDbased solution, which reduces computational complexity by finding sub-clusters of each cluster, and using their centroids to capture the structure of each cluster. This reduced problem yields much smaller matrices of which the GSVD can be applied efficiently. Experiments on text data, with up to 7000 dimensions, show that the approximation algorithm produces results that are close to those produced by the exact algorithm.

UR - http://www.scopus.com/inward/record.url?scp=78149326379&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78149326379&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:78149326379

SN - 0769519784

SN - 9780769519784

SP - 419

EP - 426

BT - Proceedings - IEEE International Conference on Data Mining, ICDM

ER -