Efficiently handling feature redundancy in high-dimensional data

Lei Yu, Huan Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

44 Scopus citations

Abstract

High-dimensional data poses a severe challenge for data mining. Feature selection is a frequently used technique in pre-processing high-dimensional data for successful data mining. Traditionally, feature selection is focused on removing irrelevant features. However, for high-dimensional data, removing redundant features is equally critical. In this paper, we provide a study of feature redundancy in high-dimensional data and propose a novel correlation-based approach to feature selection within the filter model. The extensive empirical study using real-world data shows that the proposed approach is efficient and effective in removing redundant and irrelevant features.

Original languageEnglish (US)
Title of host publicationProceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Pages685-690
Number of pages6
DOIs
StatePublished - 2003
Event9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03 - Washington, DC, United States
Duration: Aug 24 2003Aug 27 2003

Other

Other9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03
CountryUnited States
CityWashington, DC
Period8/24/038/27/03

Keywords

  • Feature selection
  • High-dimensional data
  • Redundancy

ASJC Scopus subject areas

  • Software
  • Information Systems

Fingerprint Dive into the research topics of 'Efficiently handling feature redundancy in high-dimensional data'. Together they form a unique fingerprint.

  • Cite this

    Yu, L., & Liu, H. (2003). Efficiently handling feature redundancy in high-dimensional data. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 685-690) https://doi.org/10.1145/956750.956840