Abstract

We propose a tree regularization framework, which enables many tree models to perform feature selection efficiently. The key idea of the regularization framework is to penalize selecting a new feature for splitting when its gain (e.g. information gain) is similar to the features used in previous splits. The regularization framework is applied on random forest and boosted trees here, and can be easily applied to other tree models. Experimental studies show that the regularized trees can select high-quality feature subsets with regard to both strong and weak classifiers. Because tree models can naturally deal with categorical and numerical variables, missing values, different scales between variables, interactions and nonlinearities etc., the tree regularization framework provides an effective and efficient feature selection solution for many practical problems.

Original languageEnglish (US)
Title of host publicationProceedings of the International Joint Conference on Neural Networks
DOIs
StatePublished - 2012
Event2012 Annual International Joint Conference on Neural Networks, IJCNN 2012, Part of the 2012 IEEE World Congress on Computational Intelligence, WCCI 2012 - Brisbane, QLD, Australia
Duration: Jun 10 2012Jun 15 2012

Other

Other2012 Annual International Joint Conference on Neural Networks, IJCNN 2012, Part of the 2012 IEEE World Congress on Computational Intelligence, WCCI 2012
CountryAustralia
CityBrisbane, QLD
Period6/10/126/15/12

Fingerprint

Feature extraction
Classifiers

Keywords

  • RBoost
  • regularized boosted trees
  • regularized random forest
  • RRF
  • tree regularization

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this

Deng, H., & Runger, G. (2012). Feature selection via regularized trees. In Proceedings of the International Joint Conference on Neural Networks [6252640] https://doi.org/10.1109/IJCNN.2012.6252640

Feature selection via regularized trees. / Deng, Houtao; Runger, George.

Proceedings of the International Joint Conference on Neural Networks. 2012. 6252640.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Deng, H & Runger, G 2012, Feature selection via regularized trees. in Proceedings of the International Joint Conference on Neural Networks., 6252640, 2012 Annual International Joint Conference on Neural Networks, IJCNN 2012, Part of the 2012 IEEE World Congress on Computational Intelligence, WCCI 2012, Brisbane, QLD, Australia, 6/10/12. https://doi.org/10.1109/IJCNN.2012.6252640
Deng H, Runger G. Feature selection via regularized trees. In Proceedings of the International Joint Conference on Neural Networks. 2012. 6252640 https://doi.org/10.1109/IJCNN.2012.6252640
Deng, Houtao ; Runger, George. / Feature selection via regularized trees. Proceedings of the International Joint Conference on Neural Networks. 2012.
@inproceedings{fa3915a6fdd54be384f52ab94fc3597d,
title = "Feature selection via regularized trees",
abstract = "We propose a tree regularization framework, which enables many tree models to perform feature selection efficiently. The key idea of the regularization framework is to penalize selecting a new feature for splitting when its gain (e.g. information gain) is similar to the features used in previous splits. The regularization framework is applied on random forest and boosted trees here, and can be easily applied to other tree models. Experimental studies show that the regularized trees can select high-quality feature subsets with regard to both strong and weak classifiers. Because tree models can naturally deal with categorical and numerical variables, missing values, different scales between variables, interactions and nonlinearities etc., the tree regularization framework provides an effective and efficient feature selection solution for many practical problems.",
keywords = "RBoost, regularized boosted trees, regularized random forest, RRF, tree regularization",
author = "Houtao Deng and George Runger",
year = "2012",
doi = "10.1109/IJCNN.2012.6252640",
language = "English (US)",
isbn = "9781467314909",
booktitle = "Proceedings of the International Joint Conference on Neural Networks",

}

TY - GEN

T1 - Feature selection via regularized trees

AU - Deng, Houtao

AU - Runger, George

PY - 2012

Y1 - 2012

N2 - We propose a tree regularization framework, which enables many tree models to perform feature selection efficiently. The key idea of the regularization framework is to penalize selecting a new feature for splitting when its gain (e.g. information gain) is similar to the features used in previous splits. The regularization framework is applied on random forest and boosted trees here, and can be easily applied to other tree models. Experimental studies show that the regularized trees can select high-quality feature subsets with regard to both strong and weak classifiers. Because tree models can naturally deal with categorical and numerical variables, missing values, different scales between variables, interactions and nonlinearities etc., the tree regularization framework provides an effective and efficient feature selection solution for many practical problems.

AB - We propose a tree regularization framework, which enables many tree models to perform feature selection efficiently. The key idea of the regularization framework is to penalize selecting a new feature for splitting when its gain (e.g. information gain) is similar to the features used in previous splits. The regularization framework is applied on random forest and boosted trees here, and can be easily applied to other tree models. Experimental studies show that the regularized trees can select high-quality feature subsets with regard to both strong and weak classifiers. Because tree models can naturally deal with categorical and numerical variables, missing values, different scales between variables, interactions and nonlinearities etc., the tree regularization framework provides an effective and efficient feature selection solution for many practical problems.

KW - RBoost

KW - regularized boosted trees

KW - regularized random forest

KW - RRF

KW - tree regularization

UR - http://www.scopus.com/inward/record.url?scp=84865067900&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84865067900&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2012.6252640

DO - 10.1109/IJCNN.2012.6252640

M3 - Conference contribution

AN - SCOPUS:84865067900

SN - 9781467314909

BT - Proceedings of the International Joint Conference on Neural Networks

ER -