TY - GEN
T1 - Null space versus orthogonal Linear Discriminant Analysis
AU - Ye, Jieping
AU - Xiong, Tao
N1 - Copyright:
Copyright 2008 Elsevier B.V., All rights reserved.
PY - 2006
Y1 - 2006
N2 - Dimensionality reduction is an important pre-processing step for many applications. Linear Discriminant Analysis (LDA) is one of the well known methods for supervised dimensionality reduction. However, the classical LDA formulation requires the nonsingularity of scatter matrices involved. For undersampled problems, where the data dimension is much larger than the sample size, all scatter matrices are singular and classical LDA fails. Many extensions, including null space based LDA (NLDA), orthogonal LDA (OLDA), etc, have been proposed in the past to overcome this problem. In this paper, we present a computational and theoretical analysis of NLDA and OLDA. Our main result shows that under a mild condition which holds in many applications involving high-dimensional data, NLDA is equivalent to OLDA. We have performed extensive experiments on various types of data and results are consistent with our theoretical analysis. The presented analysis and experimental results provide further insight into several LDA based algorithms.
AB - Dimensionality reduction is an important pre-processing step for many applications. Linear Discriminant Analysis (LDA) is one of the well known methods for supervised dimensionality reduction. However, the classical LDA formulation requires the nonsingularity of scatter matrices involved. For undersampled problems, where the data dimension is much larger than the sample size, all scatter matrices are singular and classical LDA fails. Many extensions, including null space based LDA (NLDA), orthogonal LDA (OLDA), etc, have been proposed in the past to overcome this problem. In this paper, we present a computational and theoretical analysis of NLDA and OLDA. Our main result shows that under a mild condition which holds in many applications involving high-dimensional data, NLDA is equivalent to OLDA. We have performed extensive experiments on various types of data and results are consistent with our theoretical analysis. The presented analysis and experimental results provide further insight into several LDA based algorithms.
UR - http://www.scopus.com/inward/record.url?scp=33749244971&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=33749244971&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:33749244971
SN - 1595933832
SN - 9781595933836
T3 - ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning
SP - 1073
EP - 1080
BT - ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning
T2 - ICML 2006: 23rd International Conference on Machine Learning
Y2 - 25 June 2006 through 29 June 2006
ER -