TY - GEN
T1 - An accelerated gradient method for trace norm minimization
AU - Ji, Shuiwang
AU - Ye, Jieping
PY - 2009
Y1 - 2009
N2 - We consider the minimization of a smooth loss function regularized by the trace norm of the matrix variable. Such formulation finds applications in many machine learning tasks including multi-task learning, matrix classification, and matrix completion. The standard semidefinite programming formulation for this problem is computationally expensive. In addition, due to the non-smooth nature of the trace norm, the optimal first-order black-box method for solving such class of problems converges as O( 1/k ), where k is the iteration counter. In this paper, we exploit the special structure of the trace norm, based on which we propose an extended gradient algorithm that converges as O( 1/k ). We further propose an accelerated gradient algorithm, which achieves the optimal convergence rate of O( 1/k2 ) for smooth problems. Experiments on multi-task learning problems demonstrate the eficiency of the proposed algorithms.
AB - We consider the minimization of a smooth loss function regularized by the trace norm of the matrix variable. Such formulation finds applications in many machine learning tasks including multi-task learning, matrix classification, and matrix completion. The standard semidefinite programming formulation for this problem is computationally expensive. In addition, due to the non-smooth nature of the trace norm, the optimal first-order black-box method for solving such class of problems converges as O( 1/k ), where k is the iteration counter. In this paper, we exploit the special structure of the trace norm, based on which we propose an extended gradient algorithm that converges as O( 1/k ). We further propose an accelerated gradient algorithm, which achieves the optimal convergence rate of O( 1/k2 ) for smooth problems. Experiments on multi-task learning problems demonstrate the eficiency of the proposed algorithms.
UR - http://www.scopus.com/inward/record.url?scp=70049092418&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=70049092418&partnerID=8YFLogxK
U2 - 10.1145/1553374.1553434
DO - 10.1145/1553374.1553434
M3 - Conference contribution
AN - SCOPUS:70049092418
SN - 9781605585161
T3 - ACM International Conference Proceeding Series
BT - Proceedings of the 26th Annual International Conference on Machine Learning, ICML'09
T2 - 26th Annual International Conference on Machine Learning, ICML'09
Y2 - 14 June 2009 through 18 June 2009
ER -