Robust feature induction for support vector machines

Rong Jin, Huan Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

The goal of feature induction is to automatically create nonlinear combinations of existing features as additional input features to improve classification accuracy. Typically, nonlinear features are introduced into a support vector machine (SVM) through a nonlinear kernel function. One disadvantage of such an approach is that the feature space induced by a kernel function is usually of high dimension and therefore will substantially increase the chance of over-fitting the training data. Another disadvantage is that nonlinear features are induced implicitly and therefore are difficult for people to understand which induced features are critical to the classification performance. In this paper, we propose a boosting-style algorithm that can explicitly induces important nonlinear features for SVMs. We present empirical studies with discussion to show that this approach is effective in improving classification accuracy for SVMs. The comparison with an SVM model using nonlinear kernels also indicates that this approach is effective and robust, particularly when the number of training data is small.

Original languageEnglish (US)
Title of host publicationProceedings, Twenty-First International Conference on Machine Learning, ICML 2004
EditorsR. Greiner, D. Schuurmans
Pages449-456
Number of pages8
StatePublished - 2004
EventProceedings, Twenty-First International Conference on Machine Learning, ICML 2004 - Banff, Alta, Canada
Duration: Jul 4 2004Jul 8 2004

Other

OtherProceedings, Twenty-First International Conference on Machine Learning, ICML 2004
CountryCanada
CityBanff, Alta
Period7/4/047/8/04

Fingerprint

Support vector machines

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Jin, R., & Liu, H. (2004). Robust feature induction for support vector machines. In R. Greiner, & D. Schuurmans (Eds.), Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004 (pp. 449-456)

Robust feature induction for support vector machines. / Jin, Rong; Liu, Huan.

Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004. ed. / R. Greiner; D. Schuurmans. 2004. p. 449-456.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Jin, R & Liu, H 2004, Robust feature induction for support vector machines. in R Greiner & D Schuurmans (eds), Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004. pp. 449-456, Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004, Banff, Alta, Canada, 7/4/04.
Jin R, Liu H. Robust feature induction for support vector machines. In Greiner R, Schuurmans D, editors, Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004. 2004. p. 449-456
Jin, Rong ; Liu, Huan. / Robust feature induction for support vector machines. Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004. editor / R. Greiner ; D. Schuurmans. 2004. pp. 449-456
@inproceedings{0eb0eef011074900b4c59c99ea8a5bfb,
title = "Robust feature induction for support vector machines",
abstract = "The goal of feature induction is to automatically create nonlinear combinations of existing features as additional input features to improve classification accuracy. Typically, nonlinear features are introduced into a support vector machine (SVM) through a nonlinear kernel function. One disadvantage of such an approach is that the feature space induced by a kernel function is usually of high dimension and therefore will substantially increase the chance of over-fitting the training data. Another disadvantage is that nonlinear features are induced implicitly and therefore are difficult for people to understand which induced features are critical to the classification performance. In this paper, we propose a boosting-style algorithm that can explicitly induces important nonlinear features for SVMs. We present empirical studies with discussion to show that this approach is effective in improving classification accuracy for SVMs. The comparison with an SVM model using nonlinear kernels also indicates that this approach is effective and robust, particularly when the number of training data is small.",
author = "Rong Jin and Huan Liu",
year = "2004",
language = "English (US)",
isbn = "1581138385",
pages = "449--456",
editor = "R. Greiner and D. Schuurmans",
booktitle = "Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004",

}

TY - GEN

T1 - Robust feature induction for support vector machines

AU - Jin, Rong

AU - Liu, Huan

PY - 2004

Y1 - 2004

N2 - The goal of feature induction is to automatically create nonlinear combinations of existing features as additional input features to improve classification accuracy. Typically, nonlinear features are introduced into a support vector machine (SVM) through a nonlinear kernel function. One disadvantage of such an approach is that the feature space induced by a kernel function is usually of high dimension and therefore will substantially increase the chance of over-fitting the training data. Another disadvantage is that nonlinear features are induced implicitly and therefore are difficult for people to understand which induced features are critical to the classification performance. In this paper, we propose a boosting-style algorithm that can explicitly induces important nonlinear features for SVMs. We present empirical studies with discussion to show that this approach is effective in improving classification accuracy for SVMs. The comparison with an SVM model using nonlinear kernels also indicates that this approach is effective and robust, particularly when the number of training data is small.

AB - The goal of feature induction is to automatically create nonlinear combinations of existing features as additional input features to improve classification accuracy. Typically, nonlinear features are introduced into a support vector machine (SVM) through a nonlinear kernel function. One disadvantage of such an approach is that the feature space induced by a kernel function is usually of high dimension and therefore will substantially increase the chance of over-fitting the training data. Another disadvantage is that nonlinear features are induced implicitly and therefore are difficult for people to understand which induced features are critical to the classification performance. In this paper, we propose a boosting-style algorithm that can explicitly induces important nonlinear features for SVMs. We present empirical studies with discussion to show that this approach is effective in improving classification accuracy for SVMs. The comparison with an SVM model using nonlinear kernels also indicates that this approach is effective and robust, particularly when the number of training data is small.

UR - http://www.scopus.com/inward/record.url?scp=14344249147&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=14344249147&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:14344249147

SN - 1581138385

SN - 9781581138382

SP - 449

EP - 456

BT - Proceedings, Twenty-First International Conference on Machine Learning, ICML 2004

A2 - Greiner, R.

A2 - Schuurmans, D.

ER -