TY - GEN
T1 - Epoch gradient descent for smoothed hinge-loss linear SVMs
AU - Lee, Soomin
AU - Nedic, Angelia
PY - 2013/9/11
Y1 - 2013/9/11
N2 - A gradient descent method for strongly convex problems with Lipschitz continuous gradients requires only O(logq ε) iterations to obtain an ε-accurate solution (q is a constant in (0; 1)). Support Vector Machines (SVMs) penalized with the popular hinge-loss are strongly convex but they do not have Lipschitz continuous gradient. We find SVMs with strong-convexity and Lipschitz continuous gradient using Nesterov's smooth approximation technique [1]. The simple gradient method applied on the smoothed SVM converges fast but the obtained solution is not the exact maximum margin separating hyperplane. To obtain an exact solution, as well as a fast convergence, we propose a hybrid approach, epoch gradient descent.
AB - A gradient descent method for strongly convex problems with Lipschitz continuous gradients requires only O(logq ε) iterations to obtain an ε-accurate solution (q is a constant in (0; 1)). Support Vector Machines (SVMs) penalized with the popular hinge-loss are strongly convex but they do not have Lipschitz continuous gradient. We find SVMs with strong-convexity and Lipschitz continuous gradient using Nesterov's smooth approximation technique [1]. The simple gradient method applied on the smoothed SVM converges fast but the obtained solution is not the exact maximum margin separating hyperplane. To obtain an exact solution, as well as a fast convergence, we propose a hybrid approach, epoch gradient descent.
UR - http://www.scopus.com/inward/record.url?scp=84883533608&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84883533608&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84883533608
SN - 9781479901777
T3 - Proceedings of the American Control Conference
SP - 4789
EP - 4794
BT - 2013 American Control Conference, ACC 2013
T2 - 2013 1st American Control Conference, ACC 2013
Y2 - 17 June 2013 through 19 June 2013
ER -