Null space versus orthogonal Linear Discriminant Analysis

Jieping Ye, Tao Xiong

Research output: Chapter in Book/Report/Conference proceedingConference contribution

21 Scopus citations

Abstract

Dimensionality reduction is an important pre-processing step for many applications. Linear Discriminant Analysis (LDA) is one of the well known methods for supervised dimensionality reduction. However, the classical LDA formulation requires the nonsingularity of scatter matrices involved. For undersampled problems, where the data dimension is much larger than the sample size, all scatter matrices are singular and classical LDA fails. Many extensions, including null space based LDA (NLDA), orthogonal LDA (OLDA), etc, have been proposed in the past to overcome this problem. In this paper, we present a computational and theoretical analysis of NLDA and OLDA. Our main result shows that under a mild condition which holds in many applications involving high-dimensional data, NLDA is equivalent to OLDA. We have performed extensive experiments on various types of data and results are consistent with our theoretical analysis. The presented analysis and experimental results provide further insight into several LDA based algorithms.

Original languageEnglish (US)
Title of host publicationICML 2006 - Proceedings of the 23rd International Conference on Machine Learning
Pages1073-1080
Number of pages8
StatePublished - Oct 6 2006
EventICML 2006: 23rd International Conference on Machine Learning - Pittsburgh, PA, United States
Duration: Jun 25 2006Jun 29 2006

Publication series

NameICML 2006 - Proceedings of the 23rd International Conference on Machine Learning
Volume2006

Other

OtherICML 2006: 23rd International Conference on Machine Learning
CountryUnited States
CityPittsburgh, PA
Period6/25/066/29/06

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint Dive into the research topics of 'Null space versus orthogonal Linear Discriminant Analysis'. Together they form a unique fingerprint.

Cite this