TY - GEN

T1 - An accelerated gradient method for trace norm minimization

AU - Ji, Shuiwang

AU - Ye, Jieping

PY - 2009/12/9

Y1 - 2009/12/9

N2 - We consider the minimization of a smooth loss function regularized by the trace norm of the matrix variable. Such formulation finds applications in many machine learning tasks including multi-task learning, matrix classification, and matrix completion. The standard semidefinite programming formulation for this problem is computationally expensive. In addition, due to the non-smooth nature of the trace norm, the optimal first-order black-box method for solving such class of problems converges as O(1/√k), where k is the iteration counter. In this paper, we exploit the special structure of the trace norm, based on which we propose an extended gradient algorithm that converges as O(1/k). We further propose an accelerated gradient algorithm, which achieves the optimal convergence rate of O(1/k2) for smooth problems. Experiments on multi-task learning problems demonstrate the efficiency of the proposed algorithms.

AB - We consider the minimization of a smooth loss function regularized by the trace norm of the matrix variable. Such formulation finds applications in many machine learning tasks including multi-task learning, matrix classification, and matrix completion. The standard semidefinite programming formulation for this problem is computationally expensive. In addition, due to the non-smooth nature of the trace norm, the optimal first-order black-box method for solving such class of problems converges as O(1/√k), where k is the iteration counter. In this paper, we exploit the special structure of the trace norm, based on which we propose an extended gradient algorithm that converges as O(1/k). We further propose an accelerated gradient algorithm, which achieves the optimal convergence rate of O(1/k2) for smooth problems. Experiments on multi-task learning problems demonstrate the efficiency of the proposed algorithms.

UR - http://www.scopus.com/inward/record.url?scp=71149103464&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=71149103464&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:71149103464

SN - 9781605585161

T3 - Proceedings of the 26th International Conference On Machine Learning, ICML 2009

SP - 457

EP - 464

BT - Proceedings of the 26th International Conference On Machine Learning, ICML 2009

T2 - 26th International Conference On Machine Learning, ICML 2009

Y2 - 14 June 2009 through 18 June 2009

ER -