Unsupervised feature selection for outlier detection by modelling hierarchical value-feature couplings

Guansong Pang, Longbing Cao, Ling Chen, Huan Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

38 Scopus citations

Abstract

Proper feature selection for unsupervised outlier detection can improve detection performance but is very challenging due to complex feature interactions, the mixture of relevant features with noisy/redundant features in imbalanced data, and the unavailability of class labels. Little work has been done on this challenge. This paper proposes a novel Coupled Unsupervised Feature Selection framework (CUFS for short) to filter out noisy or redundant features for subsequent outlier detection in categorical data. CUFS quantifies the outlierness (or relevance) of features by learning and integrating both the feature value couplings and feature couplings. Such value-To-feature couplings capture intrinsic data characteristics and distinguish relevant features from those noisy/redundant features. CUFS is further instantiated into a parameter-free Dense Subgraph-based Feature Selection method, called DSFS. We prove that DSFS retains a 2-Approximation feature subset to the optimal subset. Extensive evaluation results on 15 real-world data sets show that DSFS obtains an average 48% feature reduction rate, and enables three different types of pattern-based outlier detection methods to achieve substantially better AUC improvements and/or perform orders of magnitude faster than on the original feature set. Compared to its feature selection contender, on average, all three DSFS-based detectors achieve more than 20% AUC improvement.

Original languageEnglish (US)
Title of host publicationProceedings - 16th IEEE International Conference on Data Mining, ICDM 2016
EditorsFrancesco Bonchi, Josep Domingo-Ferrer, Ricardo Baeza-Yates, Zhi-Hua Zhou, Xindong Wu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages410-419
Number of pages10
ISBN (Electronic)9781509054725
DOIs
StatePublished - Jul 2 2016
Event16th IEEE International Conference on Data Mining, ICDM 2016 - Barcelona, Catalonia, Spain
Duration: Dec 12 2016Dec 15 2016

Publication series

NameProceedings - IEEE International Conference on Data Mining, ICDM
Volume0
ISSN (Print)1550-4786

Other

Other16th IEEE International Conference on Data Mining, ICDM 2016
Country/TerritorySpain
CityBarcelona, Catalonia
Period12/12/1612/15/16

Keywords

  • Coupling Learning
  • Non-IID Outlier Detection
  • Outlying Feature Selection

ASJC Scopus subject areas

  • General Engineering

Fingerprint

Dive into the research topics of 'Unsupervised feature selection for outlier detection by modelling hierarchical value-feature couplings'. Together they form a unique fingerprint.

Cite this