Consistency-based search in feature selection

Manoranjan Dash, Huan Liu

Research output: Contribution to journalArticle

596 Citations (Scopus)

Abstract

Feature selection is an effective technique in dealing with dimensionality reduction. For classification, it is used to find an "optimal" subset of relevant features such that the overall accuracy of classification is increased while the data size is reduced and the comprehensibility is improved. Feature selection methods contain two important aspects: evaluation of a candidate feature subset and search through the feature space. Existing algorithms adopt various measures to evaluate the goodness of feature subsets. This work focuses on inconsistency measure according to which a feature subset is inconsistent if there exist at least two instances with same feature values but with different class labels. We compare inconsistency measure with other measures and study different search strategies such as exhaustive, complete, heuristic and random search, that can be applied to this measure. We conduct an empirical study to examine the pros and cons of these search methods, give some guidelines on choosing a search method, and compare the classifier error rates before and after feature selection.

Original languageEnglish (US)
Pages (from-to)155-176
Number of pages22
JournalArtificial Intelligence
Volume151
Issue number1-2
DOIs
StatePublished - Dec 2003

Fingerprint

Feature extraction
Labels
heuristics
Classifiers
candidacy
evaluation
Feature Selection
Values
Inconsistency

Keywords

  • Branch and bound
  • Classification
  • Evaluation measures
  • Feature selection
  • Random search
  • Search strategies

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computational Theory and Mathematics

Cite this

Consistency-based search in feature selection. / Dash, Manoranjan; Liu, Huan.

In: Artificial Intelligence, Vol. 151, No. 1-2, 12.2003, p. 155-176.

Research output: Contribution to journalArticle

Dash, Manoranjan ; Liu, Huan. / Consistency-based search in feature selection. In: Artificial Intelligence. 2003 ; Vol. 151, No. 1-2. pp. 155-176.
@article{fc51c1483a77469988407b6eca4e9196,
title = "Consistency-based search in feature selection",
abstract = "Feature selection is an effective technique in dealing with dimensionality reduction. For classification, it is used to find an {"}optimal{"} subset of relevant features such that the overall accuracy of classification is increased while the data size is reduced and the comprehensibility is improved. Feature selection methods contain two important aspects: evaluation of a candidate feature subset and search through the feature space. Existing algorithms adopt various measures to evaluate the goodness of feature subsets. This work focuses on inconsistency measure according to which a feature subset is inconsistent if there exist at least two instances with same feature values but with different class labels. We compare inconsistency measure with other measures and study different search strategies such as exhaustive, complete, heuristic and random search, that can be applied to this measure. We conduct an empirical study to examine the pros and cons of these search methods, give some guidelines on choosing a search method, and compare the classifier error rates before and after feature selection.",
keywords = "Branch and bound, Classification, Evaluation measures, Feature selection, Random search, Search strategies",
author = "Manoranjan Dash and Huan Liu",
year = "2003",
month = "12",
doi = "10.1016/S0004-3702(03)00079-1",
language = "English (US)",
volume = "151",
pages = "155--176",
journal = "Artificial Intelligence",
issn = "0004-3702",
publisher = "Elsevier",
number = "1-2",

}

TY - JOUR

T1 - Consistency-based search in feature selection

AU - Dash, Manoranjan

AU - Liu, Huan

PY - 2003/12

Y1 - 2003/12

N2 - Feature selection is an effective technique in dealing with dimensionality reduction. For classification, it is used to find an "optimal" subset of relevant features such that the overall accuracy of classification is increased while the data size is reduced and the comprehensibility is improved. Feature selection methods contain two important aspects: evaluation of a candidate feature subset and search through the feature space. Existing algorithms adopt various measures to evaluate the goodness of feature subsets. This work focuses on inconsistency measure according to which a feature subset is inconsistent if there exist at least two instances with same feature values but with different class labels. We compare inconsistency measure with other measures and study different search strategies such as exhaustive, complete, heuristic and random search, that can be applied to this measure. We conduct an empirical study to examine the pros and cons of these search methods, give some guidelines on choosing a search method, and compare the classifier error rates before and after feature selection.

AB - Feature selection is an effective technique in dealing with dimensionality reduction. For classification, it is used to find an "optimal" subset of relevant features such that the overall accuracy of classification is increased while the data size is reduced and the comprehensibility is improved. Feature selection methods contain two important aspects: evaluation of a candidate feature subset and search through the feature space. Existing algorithms adopt various measures to evaluate the goodness of feature subsets. This work focuses on inconsistency measure according to which a feature subset is inconsistent if there exist at least two instances with same feature values but with different class labels. We compare inconsistency measure with other measures and study different search strategies such as exhaustive, complete, heuristic and random search, that can be applied to this measure. We conduct an empirical study to examine the pros and cons of these search methods, give some guidelines on choosing a search method, and compare the classifier error rates before and after feature selection.

KW - Branch and bound

KW - Classification

KW - Evaluation measures

KW - Feature selection

KW - Random search

KW - Search strategies

UR - http://www.scopus.com/inward/record.url?scp=0242302657&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0242302657&partnerID=8YFLogxK

U2 - 10.1016/S0004-3702(03)00079-1

DO - 10.1016/S0004-3702(03)00079-1

M3 - Article

AN - SCOPUS:0242302657

VL - 151

SP - 155

EP - 176

JO - Artificial Intelligence

JF - Artificial Intelligence

SN - 0004-3702

IS - 1-2

ER -