Multi-task feature learning via efficient ℓ2, 1-norm minimization

Jun Liu, Shuiwang Ji, Jieping Ye

Research output: Chapter in Book/Report/Conference proceedingConference contribution

445 Citations (Scopus)

Abstract

The problem of joint feature selection across a group of related tasks has applications in many areas including biomedical informatics and computer vision. We consider the ℓ2, 1-norm regularized regression model for joint feature selection from multiple tasks, which can be derived in the probabilistic framework by assuming a suitable prior from the exponential family. One appealing feature of the ℓ2, 1-norm regularization is that it encourages multiple predictors to share similar sparsity patterns. However, the resulting optimization problem is challenging to solve due to the non-smoothness of the ℓ2, 1-norm regularization. In this paper, we propose to accelerate the computation by reformulating it as two equivalent smooth convex optimization problems which are then solved via the Nesterov's method-an optimal first-order black-box method for smooth convex optimization. A key building block in solving the reformulations is the Euclidean projection. We show that the Euclidean projection for the first reformulation can be analytically computed, while the Euclidean projection for the second one can be computed in linear time. Empirical evaluations on several data sets verify the efficiency of the proposed algorithms.

Original languageEnglish (US)
Title of host publicationProceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009
Pages339-348
Number of pages10
StatePublished - 2009
Event25th Conference on Uncertainty in Artificial Intelligence, UAI 2009 - Montreal, QC, Canada
Duration: Jun 18 2009Jun 21 2009

Other

Other25th Conference on Uncertainty in Artificial Intelligence, UAI 2009
CountryCanada
CityMontreal, QC
Period6/18/096/21/09

Fingerprint

Convex optimization
Feature extraction
Euclidean
Projection
Reformulation
Convex Optimization
Norm
Feature Selection
Regularization
Computer vision
Optimization Problem
Exponential Family
Black Box
Sparsity
Computer Vision
Building Blocks
Accelerate
Linear Time
Predictors
Regression Model

ASJC Scopus subject areas

  • Artificial Intelligence
  • Applied Mathematics

Cite this

Liu, J., Ji, S., & Ye, J. (2009). Multi-task feature learning via efficient ℓ2, 1-norm minimization. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009 (pp. 339-348)

Multi-task feature learning via efficient ℓ2, 1-norm minimization. / Liu, Jun; Ji, Shuiwang; Ye, Jieping.

Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009. 2009. p. 339-348.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Liu, J, Ji, S & Ye, J 2009, Multi-task feature learning via efficient ℓ2, 1-norm minimization. in Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009. pp. 339-348, 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009, Montreal, QC, Canada, 6/18/09.
Liu J, Ji S, Ye J. Multi-task feature learning via efficient ℓ2, 1-norm minimization. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009. 2009. p. 339-348
Liu, Jun ; Ji, Shuiwang ; Ye, Jieping. / Multi-task feature learning via efficient ℓ2, 1-norm minimization. Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009. 2009. pp. 339-348
@inproceedings{97e89452e5304b28b404a1ca64dd08c1,
title = "Multi-task feature learning via efficient ℓ2, 1-norm minimization",
abstract = "The problem of joint feature selection across a group of related tasks has applications in many areas including biomedical informatics and computer vision. We consider the ℓ2, 1-norm regularized regression model for joint feature selection from multiple tasks, which can be derived in the probabilistic framework by assuming a suitable prior from the exponential family. One appealing feature of the ℓ2, 1-norm regularization is that it encourages multiple predictors to share similar sparsity patterns. However, the resulting optimization problem is challenging to solve due to the non-smoothness of the ℓ2, 1-norm regularization. In this paper, we propose to accelerate the computation by reformulating it as two equivalent smooth convex optimization problems which are then solved via the Nesterov's method-an optimal first-order black-box method for smooth convex optimization. A key building block in solving the reformulations is the Euclidean projection. We show that the Euclidean projection for the first reformulation can be analytically computed, while the Euclidean projection for the second one can be computed in linear time. Empirical evaluations on several data sets verify the efficiency of the proposed algorithms.",
author = "Jun Liu and Shuiwang Ji and Jieping Ye",
year = "2009",
language = "English (US)",
pages = "339--348",
booktitle = "Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009",

}

TY - GEN

T1 - Multi-task feature learning via efficient ℓ2, 1-norm minimization

AU - Liu, Jun

AU - Ji, Shuiwang

AU - Ye, Jieping

PY - 2009

Y1 - 2009

N2 - The problem of joint feature selection across a group of related tasks has applications in many areas including biomedical informatics and computer vision. We consider the ℓ2, 1-norm regularized regression model for joint feature selection from multiple tasks, which can be derived in the probabilistic framework by assuming a suitable prior from the exponential family. One appealing feature of the ℓ2, 1-norm regularization is that it encourages multiple predictors to share similar sparsity patterns. However, the resulting optimization problem is challenging to solve due to the non-smoothness of the ℓ2, 1-norm regularization. In this paper, we propose to accelerate the computation by reformulating it as two equivalent smooth convex optimization problems which are then solved via the Nesterov's method-an optimal first-order black-box method for smooth convex optimization. A key building block in solving the reformulations is the Euclidean projection. We show that the Euclidean projection for the first reformulation can be analytically computed, while the Euclidean projection for the second one can be computed in linear time. Empirical evaluations on several data sets verify the efficiency of the proposed algorithms.

AB - The problem of joint feature selection across a group of related tasks has applications in many areas including biomedical informatics and computer vision. We consider the ℓ2, 1-norm regularized regression model for joint feature selection from multiple tasks, which can be derived in the probabilistic framework by assuming a suitable prior from the exponential family. One appealing feature of the ℓ2, 1-norm regularization is that it encourages multiple predictors to share similar sparsity patterns. However, the resulting optimization problem is challenging to solve due to the non-smoothness of the ℓ2, 1-norm regularization. In this paper, we propose to accelerate the computation by reformulating it as two equivalent smooth convex optimization problems which are then solved via the Nesterov's method-an optimal first-order black-box method for smooth convex optimization. A key building block in solving the reformulations is the Euclidean projection. We show that the Euclidean projection for the first reformulation can be analytically computed, while the Euclidean projection for the second one can be computed in linear time. Empirical evaluations on several data sets verify the efficiency of the proposed algorithms.

UR - http://www.scopus.com/inward/record.url?scp=80053145416&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80053145416&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:80053145416

SP - 339

EP - 348

BT - Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI 2009

ER -