Multi-stage multi-task feature learning

Pinghua Gong, Jieping Ye, Changshui Zhang

Research output: Contribution to journalArticle

37 Citations (Scopus)

Abstract

Multi-task sparse feature learning aims to improve the generalization performance by exploiting the shared features among tasks. It has been successfully applied to many applications including computer vision and biomedical informatics. Most of the existing multi-task sparse feature learning algorithms are formulated as a convex sparse regularization problem, which is usually suboptimal, due to its looseness for approximating an ?0-type regularizer. In this paper, we propose a non-convex formulation for multi-task sparse feature learning based on a novel non-convex regularizer. To solve the non-convex optimization problem, we propose a Multi-Stage Multi-Task Feature Learning (MSMTFL) algorithm; we also provide intuitive interpretations, detailed convergence and reproducibility analysis for the proposed algorithm. Moreover, we present a detailed theoretical analysis showing thatMSMTFL achieves a better parameter estimation error bound than the convex formulation. Empirical studies on both synthetic and real-world data sets demonstrate the effectiveness of MSMTFL in comparison with the state of the art multi-task sparse feature learning algorithms.

Original languageEnglish (US)
Pages (from-to)2979-3010
Number of pages32
JournalJournal of Machine Learning Research
Volume14
StatePublished - Oct 2013

Fingerprint

Learning algorithms
Learning Algorithm
Parameter estimation
Computer vision
Nonconvex Optimization
Formulation
Nonconvex Problems
Reproducibility
Estimation Error
Computer Vision
Empirical Study
Error Bounds
Parameter Estimation
Intuitive
Theoretical Analysis
Regularization
Learning
Optimization Problem
Demonstrate

Keywords

  • Multi-stage
  • Multi-task learning
  • Non-convex
  • Sparse learning

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability

Cite this

Gong, P., Ye, J., & Zhang, C. (2013). Multi-stage multi-task feature learning. Journal of Machine Learning Research, 14, 2979-3010.

Multi-stage multi-task feature learning. / Gong, Pinghua; Ye, Jieping; Zhang, Changshui.

In: Journal of Machine Learning Research, Vol. 14, 10.2013, p. 2979-3010.

Research output: Contribution to journalArticle

Gong, P, Ye, J & Zhang, C 2013, 'Multi-stage multi-task feature learning', Journal of Machine Learning Research, vol. 14, pp. 2979-3010.
Gong, Pinghua ; Ye, Jieping ; Zhang, Changshui. / Multi-stage multi-task feature learning. In: Journal of Machine Learning Research. 2013 ; Vol. 14. pp. 2979-3010.
@article{9e0700bdb7314c759f8b920921003cfb,
title = "Multi-stage multi-task feature learning",
abstract = "Multi-task sparse feature learning aims to improve the generalization performance by exploiting the shared features among tasks. It has been successfully applied to many applications including computer vision and biomedical informatics. Most of the existing multi-task sparse feature learning algorithms are formulated as a convex sparse regularization problem, which is usually suboptimal, due to its looseness for approximating an ?0-type regularizer. In this paper, we propose a non-convex formulation for multi-task sparse feature learning based on a novel non-convex regularizer. To solve the non-convex optimization problem, we propose a Multi-Stage Multi-Task Feature Learning (MSMTFL) algorithm; we also provide intuitive interpretations, detailed convergence and reproducibility analysis for the proposed algorithm. Moreover, we present a detailed theoretical analysis showing thatMSMTFL achieves a better parameter estimation error bound than the convex formulation. Empirical studies on both synthetic and real-world data sets demonstrate the effectiveness of MSMTFL in comparison with the state of the art multi-task sparse feature learning algorithms.",
keywords = "Multi-stage, Multi-task learning, Non-convex, Sparse learning",
author = "Pinghua Gong and Jieping Ye and Changshui Zhang",
year = "2013",
month = "10",
language = "English (US)",
volume = "14",
pages = "2979--3010",
journal = "Journal of Machine Learning Research",
issn = "1532-4435",
publisher = "Microtome Publishing",

}

TY - JOUR

T1 - Multi-stage multi-task feature learning

AU - Gong, Pinghua

AU - Ye, Jieping

AU - Zhang, Changshui

PY - 2013/10

Y1 - 2013/10

N2 - Multi-task sparse feature learning aims to improve the generalization performance by exploiting the shared features among tasks. It has been successfully applied to many applications including computer vision and biomedical informatics. Most of the existing multi-task sparse feature learning algorithms are formulated as a convex sparse regularization problem, which is usually suboptimal, due to its looseness for approximating an ?0-type regularizer. In this paper, we propose a non-convex formulation for multi-task sparse feature learning based on a novel non-convex regularizer. To solve the non-convex optimization problem, we propose a Multi-Stage Multi-Task Feature Learning (MSMTFL) algorithm; we also provide intuitive interpretations, detailed convergence and reproducibility analysis for the proposed algorithm. Moreover, we present a detailed theoretical analysis showing thatMSMTFL achieves a better parameter estimation error bound than the convex formulation. Empirical studies on both synthetic and real-world data sets demonstrate the effectiveness of MSMTFL in comparison with the state of the art multi-task sparse feature learning algorithms.

AB - Multi-task sparse feature learning aims to improve the generalization performance by exploiting the shared features among tasks. It has been successfully applied to many applications including computer vision and biomedical informatics. Most of the existing multi-task sparse feature learning algorithms are formulated as a convex sparse regularization problem, which is usually suboptimal, due to its looseness for approximating an ?0-type regularizer. In this paper, we propose a non-convex formulation for multi-task sparse feature learning based on a novel non-convex regularizer. To solve the non-convex optimization problem, we propose a Multi-Stage Multi-Task Feature Learning (MSMTFL) algorithm; we also provide intuitive interpretations, detailed convergence and reproducibility analysis for the proposed algorithm. Moreover, we present a detailed theoretical analysis showing thatMSMTFL achieves a better parameter estimation error bound than the convex formulation. Empirical studies on both synthetic and real-world data sets demonstrate the effectiveness of MSMTFL in comparison with the state of the art multi-task sparse feature learning algorithms.

KW - Multi-stage

KW - Multi-task learning

KW - Non-convex

KW - Sparse learning

UR - http://www.scopus.com/inward/record.url?scp=84887435667&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84887435667&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:84887435667

VL - 14

SP - 2979

EP - 3010

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -