Feature reduction via generalized uncorrelated linear discriminant analysis

Jieping Ye, Ravi Janardan, Qi Li, Haesun Park

Research output: Contribution to journalArticle

93 Scopus citations

Abstract

High-dimensional data appear in many applications of data mining, machine learning, and bioinformatics. Feature reduction is commonly applied as a preprocessing step to overcome the curse of dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for feature reduction. The extracted features via ULDA were shown to be statistically uncorrelated, which is desirable for many applications. In this paper, an algorithm called ULDA/QR is proposed to simplify the previous implementation of ULDA. Then, the ULDA/ GSVD algorithm is proposed, based on a novel optimization criterion, to address the singularity problem which occurs in undersampled problems, where the data dimension is larger than the sample size. The criterion used is the regularized version of the one in ULDA/QR. Surprisingly, our theoretical result shows that the solution to ULDA/GSVD is independent of the value of the regularization parameter. Experimental results on various types of data sets are reported to show the effectiveness of the proposed algorithm and to compare it with other commonly used feature reduction algorithms.

Original languageEnglish (US)
Article number1683768
Pages (from-to)1312-1322
Number of pages11
JournalIEEE Transactions on Knowledge and Data Engineering
Volume18
Issue number10
DOIs
StatePublished - Oct 1 2006

    Fingerprint

Keywords

  • Feature reduction
  • Generalized singular value decomposition
  • QR-decomposition
  • Uncorrelated linear discriminant analysis

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Cite this