Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems

Jieping Ye

Research output: Contribution to journalArticle

285 Citations (Scopus)

Abstract

A generalized discriminant analysis based on a new optimization criterion is presented. The criterion extends the optimization criteria of the classical Linear Discriminant Analysis (LDA) when the scatter matrices are singular. An efficient algorithm for the new optimization problem is presented. The solutions to the proposed criterion form a family of algorithms for generalized LDA, which can be characterized in a closed form. We study two specific algorithms, namely Uncorrelated LDA (ULDA) and Orthogonal LDA (OLDA). ULDA was previously proposed for feature extraction and dimension reduction, whereas OLDA is a novel algorithm proposed in this paper. The features in the reduced space of ULDA are uncorrelated, while the discriminant vectors of OLDA are orthogonal to each other. We have conducted a comparative study on a variety of real-world data sets to evaluate ULDA and OLDA in terms of classification accuracy.

Original languageEnglish (US)
JournalJournal of Machine Learning Research
Volume6
StatePublished - 2005
Externally publishedYes

Fingerprint

Discriminant analysis
Discriminant Analysis
Optimization
Dimension Reduction
Scatter
Discriminant
Feature Extraction
Comparative Study
Feature extraction
Closed-form
Efficient Algorithms
Family
Optimization Problem
Evaluate

Keywords

  • Dimension reduction
  • Linear discriminant analysis
  • Orthogonal LDA
  • Singular value decomposition
  • Uncorrelated LDA

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

@article{510241b14ec747059772f6c14c1b5066,
title = "Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems",
abstract = "A generalized discriminant analysis based on a new optimization criterion is presented. The criterion extends the optimization criteria of the classical Linear Discriminant Analysis (LDA) when the scatter matrices are singular. An efficient algorithm for the new optimization problem is presented. The solutions to the proposed criterion form a family of algorithms for generalized LDA, which can be characterized in a closed form. We study two specific algorithms, namely Uncorrelated LDA (ULDA) and Orthogonal LDA (OLDA). ULDA was previously proposed for feature extraction and dimension reduction, whereas OLDA is a novel algorithm proposed in this paper. The features in the reduced space of ULDA are uncorrelated, while the discriminant vectors of OLDA are orthogonal to each other. We have conducted a comparative study on a variety of real-world data sets to evaluate ULDA and OLDA in terms of classification accuracy.",
keywords = "Dimension reduction, Linear discriminant analysis, Orthogonal LDA, Singular value decomposition, Uncorrelated LDA",
author = "Jieping Ye",
year = "2005",
language = "English (US)",
volume = "6",
journal = "Journal of Machine Learning Research",
issn = "1532-4435",
publisher = "Microtome Publishing",

}

TY - JOUR

T1 - Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems

AU - Ye, Jieping

PY - 2005

Y1 - 2005

N2 - A generalized discriminant analysis based on a new optimization criterion is presented. The criterion extends the optimization criteria of the classical Linear Discriminant Analysis (LDA) when the scatter matrices are singular. An efficient algorithm for the new optimization problem is presented. The solutions to the proposed criterion form a family of algorithms for generalized LDA, which can be characterized in a closed form. We study two specific algorithms, namely Uncorrelated LDA (ULDA) and Orthogonal LDA (OLDA). ULDA was previously proposed for feature extraction and dimension reduction, whereas OLDA is a novel algorithm proposed in this paper. The features in the reduced space of ULDA are uncorrelated, while the discriminant vectors of OLDA are orthogonal to each other. We have conducted a comparative study on a variety of real-world data sets to evaluate ULDA and OLDA in terms of classification accuracy.

AB - A generalized discriminant analysis based on a new optimization criterion is presented. The criterion extends the optimization criteria of the classical Linear Discriminant Analysis (LDA) when the scatter matrices are singular. An efficient algorithm for the new optimization problem is presented. The solutions to the proposed criterion form a family of algorithms for generalized LDA, which can be characterized in a closed form. We study two specific algorithms, namely Uncorrelated LDA (ULDA) and Orthogonal LDA (OLDA). ULDA was previously proposed for feature extraction and dimension reduction, whereas OLDA is a novel algorithm proposed in this paper. The features in the reduced space of ULDA are uncorrelated, while the discriminant vectors of OLDA are orthogonal to each other. We have conducted a comparative study on a variety of real-world data sets to evaluate ULDA and OLDA in terms of classification accuracy.

KW - Dimension reduction

KW - Linear discriminant analysis

KW - Orthogonal LDA

KW - Singular value decomposition

KW - Uncorrelated LDA

UR - http://www.scopus.com/inward/record.url?scp=21844447839&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=21844447839&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:21844447839

VL - 6

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -