TY - JOUR
T1 - Feature reduction via generalized uncorrelated linear discriminant analysis
AU - Ye, Jieping
AU - Janardan, Ravi
AU - Li, Qi
AU - Park, Haesun
N1 - Funding Information:
endorsement should be inferred. Fellowships from Guidant Corporation and from the Department of Computer Science and Engineering, at the University of Minnesota, Twin Cities are gratefully acknowledged. The work of H. Park has been performed while serving as a program director at the US National Science Foundation (NSF) and was partly supported by IR/D from the NSF. Her work was also supported in part by the US National Science Foundation Grants CCR-0204109 and ACI-0305543. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the US National Science Foundation.
Funding Information:
The authors would like to thank the four reviewers and the associate editor for their comments, which helped improve the paper significantly. The research of J. Ye and R. Janardan was sponsored, in part, by the Army High Performance Computing Research Center under the auspices of the Department of the Army, Army Research Laboratory cooperative agreement number DAAD19-01-2-0014, the content of which does not necessarily reflect the position or the policy of the government, and no official
PY - 2006/10
Y1 - 2006/10
N2 - High-dimensional data appear in many applications of data mining, machine learning, and bioinformatics. Feature reduction is commonly applied as a preprocessing step to overcome the curse of dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for feature reduction. The extracted features via ULDA were shown to be statistically uncorrelated, which is desirable for many applications. In this paper, an algorithm called ULDA/QR is proposed to simplify the previous implementation of ULDA. Then, the ULDA/ GSVD algorithm is proposed, based on a novel optimization criterion, to address the singularity problem which occurs in undersampled problems, where the data dimension is larger than the sample size. The criterion used is the regularized version of the one in ULDA/QR. Surprisingly, our theoretical result shows that the solution to ULDA/GSVD is independent of the value of the regularization parameter. Experimental results on various types of data sets are reported to show the effectiveness of the proposed algorithm and to compare it with other commonly used feature reduction algorithms.
AB - High-dimensional data appear in many applications of data mining, machine learning, and bioinformatics. Feature reduction is commonly applied as a preprocessing step to overcome the curse of dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for feature reduction. The extracted features via ULDA were shown to be statistically uncorrelated, which is desirable for many applications. In this paper, an algorithm called ULDA/QR is proposed to simplify the previous implementation of ULDA. Then, the ULDA/ GSVD algorithm is proposed, based on a novel optimization criterion, to address the singularity problem which occurs in undersampled problems, where the data dimension is larger than the sample size. The criterion used is the regularized version of the one in ULDA/QR. Surprisingly, our theoretical result shows that the solution to ULDA/GSVD is independent of the value of the regularization parameter. Experimental results on various types of data sets are reported to show the effectiveness of the proposed algorithm and to compare it with other commonly used feature reduction algorithms.
KW - Feature reduction
KW - Generalized singular value decomposition
KW - QR-decomposition
KW - Uncorrelated linear discriminant analysis
UR - http://www.scopus.com/inward/record.url?scp=33748336501&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=33748336501&partnerID=8YFLogxK
U2 - 10.1109/TKDE.2006.160
DO - 10.1109/TKDE.2006.160
M3 - Article
AN - SCOPUS:33748336501
SN - 1041-4347
VL - 18
SP - 1312
EP - 1322
JO - IEEE Transactions on Knowledge and Data Engineering
JF - IEEE Transactions on Knowledge and Data Engineering
IS - 10
M1 - 1683768
ER -