Epoch gradient descent for smoothed hinge-loss linear SVMs

Soomin Lee, Angelia Nedic

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A gradient descent method for strongly convex problems with Lipschitz continuous gradients requires only O(logq ε) iterations to obtain an ε-accurate solution (q is a constant in (0; 1)). Support Vector Machines (SVMs) penalized with the popular hinge-loss are strongly convex but they do not have Lipschitz continuous gradient. We find SVMs with strong-convexity and Lipschitz continuous gradient using Nesterov's smooth approximation technique [1]. The simple gradient method applied on the smoothed SVM converges fast but the obtained solution is not the exact maximum margin separating hyperplane. To obtain an exact solution, as well as a fast convergence, we propose a hybrid approach, epoch gradient descent.

Original languageEnglish (US)
Title of host publication2013 American Control Conference, ACC 2013
Pages4789-4794
Number of pages6
StatePublished - Sep 11 2013
Externally publishedYes
Event2013 1st American Control Conference, ACC 2013 - Washington, DC, United States
Duration: Jun 17 2013Jun 19 2013

Publication series

NameProceedings of the American Control Conference
ISSN (Print)0743-1619

Other

Other2013 1st American Control Conference, ACC 2013
CountryUnited States
CityWashington, DC
Period6/17/136/19/13

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Epoch gradient descent for smoothed hinge-loss linear SVMs'. Together they form a unique fingerprint.

Cite this