Epoch gradient descent for smoothed hinge-loss linear SVMs

Soomin Lee, Angelia Nedic

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A gradient descent method for strongly convex problems with Lipschitz continuous gradients requires only O(logq ε) iterations to obtain an ε-accurate solution (q is a constant in (0; 1)). Support Vector Machines (SVMs) penalized with the popular hinge-loss are strongly convex but they do not have Lipschitz continuous gradient. We find SVMs with strong-convexity and Lipschitz continuous gradient using Nesterov's smooth approximation technique [1]. The simple gradient method applied on the smoothed SVM converges fast but the obtained solution is not the exact maximum margin separating hyperplane. To obtain an exact solution, as well as a fast convergence, we propose a hybrid approach, epoch gradient descent.

Original languageEnglish (US)
Title of host publication2013 American Control Conference, ACC 2013
Pages4789-4794
Number of pages6
StatePublished - Sep 11 2013
Event2013 1st American Control Conference, ACC 2013 - Washington, DC, United States
Duration: Jun 17 2013Jun 19 2013

Publication series

NameProceedings of the American Control Conference
ISSN (Print)0743-1619

Other

Other2013 1st American Control Conference, ACC 2013
CountryUnited States
CityWashington, DC
Period6/17/136/19/13

    Fingerprint

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Lee, S., & Nedic, A. (2013). Epoch gradient descent for smoothed hinge-loss linear SVMs. In 2013 American Control Conference, ACC 2013 (pp. 4789-4794). [6580579] (Proceedings of the American Control Conference).