Scaling SVM and least absolute deviations via exact data reduction

Jie Wang, Peter Wonka, Jieping Ye

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Citations (Scopus)

Abstract

The support vector machine (SVM) is a widely used method for classification. Although many efforts have been devoted to develop efficient solvers, it remains challenging to apply SVM to large-scale problems. A nice property of SVM is that the non-support vectors have no effect on the resulting classifier. Motivated by this observation, we present fast and efficient screening rules to discard non-support vectors by analyzing the dual problem of SVM via variational inequalities (DVI). As a result, the number of data instances to be entered into the optimization can be substantially reduced. Some appealing features of our screening method are: (1) DVI is safe in the sense that the vectors discarded by DVI are guaranteed to be non-support vectors; (2) the data set needs to be scanned only once to run the screening, and its computational cost is negligible compared to that of solving the SVM problem; (3) DVI is independent of the solvers and can be integrated with any existing efficient solver. We also show that the DVI technique can be extended to detect non-support vectors in the least absolute deviations regression (LAD). To the best of our knowledge, there are currently no screening methods for LAD. We have evaluated DVI on both synthetic and real data sets. Experiments indicate that DVI significantly outperforms the existing state-of-the-art screening rules for SVM, and it is very effective in discarding non-support vectors for LAD. The speedup gained by DVI rules can be up to two orders of magnitude.

Original languageEnglish (US)
Title of host publication31st International Conference on Machine Learning, ICML 2014
PublisherInternational Machine Learning Society (IMLS)
Pages1912-1927
Number of pages16
Volume3
ISBN (Print)9781634393973
StatePublished - 2014
Externally publishedYes
Event31st International Conference on Machine Learning, ICML 2014 - Beijing, China
Duration: Jun 21 2014Jun 26 2014

Other

Other31st International Conference on Machine Learning, ICML 2014
CountryChina
CityBeijing
Period6/21/146/26/14

Fingerprint

Support vector machines
Data reduction
Screening
Classifiers
Costs
Experiments

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Software

Cite this

Wang, J., Wonka, P., & Ye, J. (2014). Scaling SVM and least absolute deviations via exact data reduction. In 31st International Conference on Machine Learning, ICML 2014 (Vol. 3, pp. 1912-1927). International Machine Learning Society (IMLS).

Scaling SVM and least absolute deviations via exact data reduction. / Wang, Jie; Wonka, Peter; Ye, Jieping.

31st International Conference on Machine Learning, ICML 2014. Vol. 3 International Machine Learning Society (IMLS), 2014. p. 1912-1927.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, J, Wonka, P & Ye, J 2014, Scaling SVM and least absolute deviations via exact data reduction. in 31st International Conference on Machine Learning, ICML 2014. vol. 3, International Machine Learning Society (IMLS), pp. 1912-1927, 31st International Conference on Machine Learning, ICML 2014, Beijing, China, 6/21/14.
Wang J, Wonka P, Ye J. Scaling SVM and least absolute deviations via exact data reduction. In 31st International Conference on Machine Learning, ICML 2014. Vol. 3. International Machine Learning Society (IMLS). 2014. p. 1912-1927
Wang, Jie ; Wonka, Peter ; Ye, Jieping. / Scaling SVM and least absolute deviations via exact data reduction. 31st International Conference on Machine Learning, ICML 2014. Vol. 3 International Machine Learning Society (IMLS), 2014. pp. 1912-1927
@inproceedings{a18c9dd1081e40789ecafba61b4ccf58,
title = "Scaling SVM and least absolute deviations via exact data reduction",
abstract = "The support vector machine (SVM) is a widely used method for classification. Although many efforts have been devoted to develop efficient solvers, it remains challenging to apply SVM to large-scale problems. A nice property of SVM is that the non-support vectors have no effect on the resulting classifier. Motivated by this observation, we present fast and efficient screening rules to discard non-support vectors by analyzing the dual problem of SVM via variational inequalities (DVI). As a result, the number of data instances to be entered into the optimization can be substantially reduced. Some appealing features of our screening method are: (1) DVI is safe in the sense that the vectors discarded by DVI are guaranteed to be non-support vectors; (2) the data set needs to be scanned only once to run the screening, and its computational cost is negligible compared to that of solving the SVM problem; (3) DVI is independent of the solvers and can be integrated with any existing efficient solver. We also show that the DVI technique can be extended to detect non-support vectors in the least absolute deviations regression (LAD). To the best of our knowledge, there are currently no screening methods for LAD. We have evaluated DVI on both synthetic and real data sets. Experiments indicate that DVI significantly outperforms the existing state-of-the-art screening rules for SVM, and it is very effective in discarding non-support vectors for LAD. The speedup gained by DVI rules can be up to two orders of magnitude.",
author = "Jie Wang and Peter Wonka and Jieping Ye",
year = "2014",
language = "English (US)",
isbn = "9781634393973",
volume = "3",
pages = "1912--1927",
booktitle = "31st International Conference on Machine Learning, ICML 2014",
publisher = "International Machine Learning Society (IMLS)",

}

TY - GEN

T1 - Scaling SVM and least absolute deviations via exact data reduction

AU - Wang, Jie

AU - Wonka, Peter

AU - Ye, Jieping

PY - 2014

Y1 - 2014

N2 - The support vector machine (SVM) is a widely used method for classification. Although many efforts have been devoted to develop efficient solvers, it remains challenging to apply SVM to large-scale problems. A nice property of SVM is that the non-support vectors have no effect on the resulting classifier. Motivated by this observation, we present fast and efficient screening rules to discard non-support vectors by analyzing the dual problem of SVM via variational inequalities (DVI). As a result, the number of data instances to be entered into the optimization can be substantially reduced. Some appealing features of our screening method are: (1) DVI is safe in the sense that the vectors discarded by DVI are guaranteed to be non-support vectors; (2) the data set needs to be scanned only once to run the screening, and its computational cost is negligible compared to that of solving the SVM problem; (3) DVI is independent of the solvers and can be integrated with any existing efficient solver. We also show that the DVI technique can be extended to detect non-support vectors in the least absolute deviations regression (LAD). To the best of our knowledge, there are currently no screening methods for LAD. We have evaluated DVI on both synthetic and real data sets. Experiments indicate that DVI significantly outperforms the existing state-of-the-art screening rules for SVM, and it is very effective in discarding non-support vectors for LAD. The speedup gained by DVI rules can be up to two orders of magnitude.

AB - The support vector machine (SVM) is a widely used method for classification. Although many efforts have been devoted to develop efficient solvers, it remains challenging to apply SVM to large-scale problems. A nice property of SVM is that the non-support vectors have no effect on the resulting classifier. Motivated by this observation, we present fast and efficient screening rules to discard non-support vectors by analyzing the dual problem of SVM via variational inequalities (DVI). As a result, the number of data instances to be entered into the optimization can be substantially reduced. Some appealing features of our screening method are: (1) DVI is safe in the sense that the vectors discarded by DVI are guaranteed to be non-support vectors; (2) the data set needs to be scanned only once to run the screening, and its computational cost is negligible compared to that of solving the SVM problem; (3) DVI is independent of the solvers and can be integrated with any existing efficient solver. We also show that the DVI technique can be extended to detect non-support vectors in the least absolute deviations regression (LAD). To the best of our knowledge, there are currently no screening methods for LAD. We have evaluated DVI on both synthetic and real data sets. Experiments indicate that DVI significantly outperforms the existing state-of-the-art screening rules for SVM, and it is very effective in discarding non-support vectors for LAD. The speedup gained by DVI rules can be up to two orders of magnitude.

UR - http://www.scopus.com/inward/record.url?scp=84919949965&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84919949965&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84919949965

SN - 9781634393973

VL - 3

SP - 1912

EP - 1927

BT - 31st International Conference on Machine Learning, ICML 2014

PB - International Machine Learning Society (IMLS)

ER -