Abstract

Proper feature selection for unsupervised outlier detection can improve detection performance but is very challenging due to complex feature interactions, the mixture of relevant features with noisy/redundant features in imbalanced data, and the unavailability of class labels. Little work has been done on this challenge. This paper proposes a novel Coupled Unsupervised Feature Selection framework (CUFS for short) to filter out noisy or redundant features for subsequent outlier detection in categorical data. CUFS quantifies the outlierness (or relevance) of features by learning and integrating both the feature value couplings and feature couplings. Such value-To-feature couplings capture intrinsic data characteristics and distinguish relevant features from those noisy/redundant features. CUFS is further instantiated into a parameter-free Dense Subgraph-based Feature Selection method, called DSFS. We prove that DSFS retains a 2-Approximation feature subset to the optimal subset. Extensive evaluation results on 15 real-world data sets show that DSFS obtains an average 48% feature reduction rate, and enables three different types of pattern-based outlier detection methods to achieve substantially better AUC improvements and/or perform orders of magnitude faster than on the original feature set. Compared to its feature selection contender, on average, all three DSFS-based detectors achieve more than 20% AUC improvement.

Original languageEnglish (US)
Title of host publicationProceedings - 16th IEEE International Conference on Data Mining, ICDM 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages410-419
Number of pages10
ISBN (Electronic)9781509054725
DOIs
StatePublished - Jan 31 2017
Event16th IEEE International Conference on Data Mining, ICDM 2016 - Barcelona, Catalonia, Spain
Duration: Dec 12 2016Dec 15 2016

Other

Other16th IEEE International Conference on Data Mining, ICDM 2016
CountrySpain
CityBarcelona, Catalonia
Period12/12/1612/15/16

Keywords

  • Coupling Learning
  • Non-IID Outlier Detection
  • Outlying Feature Selection

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint Dive into the research topics of 'Unsupervised feature selection for outlier detection by modelling hierarchical value-feature couplings'. Together they form a unique fingerprint.

  • Cite this

    Pang, G., Cao, L., Chen, L., & Liu, H. (2017). Unsupervised feature selection for outlier detection by modelling hierarchical value-feature couplings. In Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016 (pp. 410-419). [7837865] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDM.2016.30