A scalable two-stage approach for a class of dimensionality reduction techniques

Liang Sun, Betul Ceran, Jieping Ye

Research output: Chapter in Book/Report/Conference proceedingConference contribution

27 Citations (Scopus)

Abstract

Dimensionality reduction plays an important role in many data mining applications involving high-dimensional data. Many existing dimensionality reduction techniques can be formulated as a generalized eigenvalue problem, which does not scale to large-size problems. Prior work transforms the generalized eigenvalue problem into an equivalent least squares formulation, which can then be solved efficiently. However, the equivalence relationship only holds under certain assumptions without regularization, which severely limits their applicability in practice. In this paper, an efficient two-stage approach is proposed to solve a class of dimensionality reduction techniques, including Canonical Correlation Analysis, Orthonormal Partial Least Squares, Linear Discriminant Analysis, and Hypergraph Spectral Learning. The proposed two-stage approach scales linearly in terms of both the sample size and data dimensionality. The main contributions of this paper include (1) we rigorously establish the equivalence relationship between the proposed two-stage approach and the original formulation without any assumption; and (2) we show that the equivalence relationship still holds in the regularization setting. We have conducted extensive experiments using both synthetic and real-world data sets. Our experimental results confirm the equivalence relationship established in this paper. Results also demonstrate the scalability of the proposed two-stage approach.

Original languageEnglish (US)
Title of host publicationProceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Pages313-322
Number of pages10
DOIs
StatePublished - 2010
Event16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD-2010 - Washington, DC, United States
Duration: Jul 25 2010Jul 28 2010

Other

Other16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD-2010
CountryUnited States
CityWashington, DC
Period7/25/107/28/10

Fingerprint

Discriminant analysis
Data mining
Scalability
Experiments

Keywords

  • Dimensionality reduction
  • Generalized eigenvalue problem
  • Least squares
  • Regularization
  • Scalability

ASJC Scopus subject areas

  • Software
  • Information Systems

Cite this

Sun, L., Ceran, B., & Ye, J. (2010). A scalable two-stage approach for a class of dimensionality reduction techniques. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 313-322) https://doi.org/10.1145/1835804.1835846

A scalable two-stage approach for a class of dimensionality reduction techniques. / Sun, Liang; Ceran, Betul; Ye, Jieping.

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2010. p. 313-322.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sun, L, Ceran, B & Ye, J 2010, A scalable two-stage approach for a class of dimensionality reduction techniques. in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pp. 313-322, 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD-2010, Washington, DC, United States, 7/25/10. https://doi.org/10.1145/1835804.1835846
Sun L, Ceran B, Ye J. A scalable two-stage approach for a class of dimensionality reduction techniques. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2010. p. 313-322 https://doi.org/10.1145/1835804.1835846
Sun, Liang ; Ceran, Betul ; Ye, Jieping. / A scalable two-stage approach for a class of dimensionality reduction techniques. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2010. pp. 313-322
@inproceedings{f0f3bf21e8394d8db241209048e77192,
title = "A scalable two-stage approach for a class of dimensionality reduction techniques",
abstract = "Dimensionality reduction plays an important role in many data mining applications involving high-dimensional data. Many existing dimensionality reduction techniques can be formulated as a generalized eigenvalue problem, which does not scale to large-size problems. Prior work transforms the generalized eigenvalue problem into an equivalent least squares formulation, which can then be solved efficiently. However, the equivalence relationship only holds under certain assumptions without regularization, which severely limits their applicability in practice. In this paper, an efficient two-stage approach is proposed to solve a class of dimensionality reduction techniques, including Canonical Correlation Analysis, Orthonormal Partial Least Squares, Linear Discriminant Analysis, and Hypergraph Spectral Learning. The proposed two-stage approach scales linearly in terms of both the sample size and data dimensionality. The main contributions of this paper include (1) we rigorously establish the equivalence relationship between the proposed two-stage approach and the original formulation without any assumption; and (2) we show that the equivalence relationship still holds in the regularization setting. We have conducted extensive experiments using both synthetic and real-world data sets. Our experimental results confirm the equivalence relationship established in this paper. Results also demonstrate the scalability of the proposed two-stage approach.",
keywords = "Dimensionality reduction, Generalized eigenvalue problem, Least squares, Regularization, Scalability",
author = "Liang Sun and Betul Ceran and Jieping Ye",
year = "2010",
doi = "10.1145/1835804.1835846",
language = "English (US)",
isbn = "9781450300551",
pages = "313--322",
booktitle = "Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",

}

TY - GEN

T1 - A scalable two-stage approach for a class of dimensionality reduction techniques

AU - Sun, Liang

AU - Ceran, Betul

AU - Ye, Jieping

PY - 2010

Y1 - 2010

N2 - Dimensionality reduction plays an important role in many data mining applications involving high-dimensional data. Many existing dimensionality reduction techniques can be formulated as a generalized eigenvalue problem, which does not scale to large-size problems. Prior work transforms the generalized eigenvalue problem into an equivalent least squares formulation, which can then be solved efficiently. However, the equivalence relationship only holds under certain assumptions without regularization, which severely limits their applicability in practice. In this paper, an efficient two-stage approach is proposed to solve a class of dimensionality reduction techniques, including Canonical Correlation Analysis, Orthonormal Partial Least Squares, Linear Discriminant Analysis, and Hypergraph Spectral Learning. The proposed two-stage approach scales linearly in terms of both the sample size and data dimensionality. The main contributions of this paper include (1) we rigorously establish the equivalence relationship between the proposed two-stage approach and the original formulation without any assumption; and (2) we show that the equivalence relationship still holds in the regularization setting. We have conducted extensive experiments using both synthetic and real-world data sets. Our experimental results confirm the equivalence relationship established in this paper. Results also demonstrate the scalability of the proposed two-stage approach.

AB - Dimensionality reduction plays an important role in many data mining applications involving high-dimensional data. Many existing dimensionality reduction techniques can be formulated as a generalized eigenvalue problem, which does not scale to large-size problems. Prior work transforms the generalized eigenvalue problem into an equivalent least squares formulation, which can then be solved efficiently. However, the equivalence relationship only holds under certain assumptions without regularization, which severely limits their applicability in practice. In this paper, an efficient two-stage approach is proposed to solve a class of dimensionality reduction techniques, including Canonical Correlation Analysis, Orthonormal Partial Least Squares, Linear Discriminant Analysis, and Hypergraph Spectral Learning. The proposed two-stage approach scales linearly in terms of both the sample size and data dimensionality. The main contributions of this paper include (1) we rigorously establish the equivalence relationship between the proposed two-stage approach and the original formulation without any assumption; and (2) we show that the equivalence relationship still holds in the regularization setting. We have conducted extensive experiments using both synthetic and real-world data sets. Our experimental results confirm the equivalence relationship established in this paper. Results also demonstrate the scalability of the proposed two-stage approach.

KW - Dimensionality reduction

KW - Generalized eigenvalue problem

KW - Least squares

KW - Regularization

KW - Scalability

UR - http://www.scopus.com/inward/record.url?scp=77956202761&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77956202761&partnerID=8YFLogxK

U2 - 10.1145/1835804.1835846

DO - 10.1145/1835804.1835846

M3 - Conference contribution

SN - 9781450300551

SP - 313

EP - 322

BT - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

ER -