Learning incoherent sparse and low-rank patterns from multiple tasks

Jianhui Chen, Ji Liu, Jieping Ye

Research output: Contribution to journalArticlepeer-review

104 Scopus citations

Abstract

We consider the problem of learning incoherent sparse and low-rank patterns from multiple tasks. Our approach is based on a linear multitask learning formulation, in which the sparse and low-rank patterns are induced by a cardinality regularization term and a low-rank constraint, respectively. This formulation is nonconvex; we convert it into its convex surrogate, which can be routinely solved via semidefinite programming for small-size problems. We propose employing the general projected gradient scheme to efficiently solve such a convex surrogate; however, in the optimization formulation, the objective function is nondifferentiable and the feasible domain is nontrivial. We present the procedures for computing the projected gradient and ensuring the global convergence of the projected gradient scheme. The computation of the projected gradient involves a constrained optimization problem; we show that the optimal solution to such a problem can be obtained via solving an unconstrained optimization subproblem and a Euclidean projection subproblem. We also present two projected gradient algorithms and analyze their rates of convergence in detail. In addition, we illustrate the use of the presented projected gradient algorithms for the proposed multitask learning formulation using the least squares loss. Experimental results on a collection of real-world data sets demonstrate the effectiveness of the proposed multitask learning formulation and the efficiency of the proposed projected gradient algorithms.

Original languageEnglish (US)
Article number22
JournalACM Transactions on Knowledge Discovery from Data
Volume5
Issue number4
DOIs
StatePublished - Feb 2012

Keywords

  • Low-rank and sparse patterns
  • Multitask learning
  • Trace norm

ASJC Scopus subject areas

  • General Computer Science

Fingerprint

Dive into the research topics of 'Learning incoherent sparse and low-rank patterns from multiple tasks'. Together they form a unique fingerprint.

Cite this