Sparse trace norm regularization

Jianhui Chen, Jieping Ye

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

We study the problem of estimating multiple predictive functions from a dictionary of basis functions in the nonparametric regression setting. Our estimation scheme assumes that each predictive function can be estimated in the form of a linear combination of the basis functions. By assuming that the coefficient matrix admits a sparse low-rank structure, we formulate the function estimation problem as a convex program regularized by the trace norm and the ℓ1-norm simultaneously. We propose to solve the convex program using the accelerated gradient (AG) method; we also develop efficient algorithms to solve the key components in AG. In addition, we conduct theoretical analysis on the proposed function estimation scheme: we derive a key property of the optimal solution to the convex program; based on an assumption on the basis functions, we establish a performance bound of the proposed function estimation scheme (via the composite regularization). Simulation studies demonstrate the effectiveness and efficiency of the proposed algorithms.

Original languageEnglish (US)
Pages (from-to)623-639
Number of pages17
JournalComputational Statistics
Volume29
Issue number3-4
DOIs
StatePublished - 2014

Fingerprint

Convex Program
Function Estimation
Basis Functions
Regularization
Trace
Norm
Performance Bounds
Gradient Method
Nonparametric Regression
Linear Combination
Theoretical Analysis
Efficient Algorithms
Optimal Solution
Composite
Simulation Study
Gradient
Coefficient
Demonstrate
Gradient methods
Glossaries

Keywords

  • Gradient method
  • Low Rank
  • Performance bound
  • Regression

ASJC Scopus subject areas

  • Statistics and Probability
  • Computational Mathematics
  • Statistics, Probability and Uncertainty

Cite this

Sparse trace norm regularization. / Chen, Jianhui; Ye, Jieping.

In: Computational Statistics, Vol. 29, No. 3-4, 2014, p. 623-639.

Research output: Contribution to journalArticle

Chen, Jianhui ; Ye, Jieping. / Sparse trace norm regularization. In: Computational Statistics. 2014 ; Vol. 29, No. 3-4. pp. 623-639.
@article{9659424b709c4a52bf1c1359c33685d1,
title = "Sparse trace norm regularization",
abstract = "We study the problem of estimating multiple predictive functions from a dictionary of basis functions in the nonparametric regression setting. Our estimation scheme assumes that each predictive function can be estimated in the form of a linear combination of the basis functions. By assuming that the coefficient matrix admits a sparse low-rank structure, we formulate the function estimation problem as a convex program regularized by the trace norm and the ℓ1-norm simultaneously. We propose to solve the convex program using the accelerated gradient (AG) method; we also develop efficient algorithms to solve the key components in AG. In addition, we conduct theoretical analysis on the proposed function estimation scheme: we derive a key property of the optimal solution to the convex program; based on an assumption on the basis functions, we establish a performance bound of the proposed function estimation scheme (via the composite regularization). Simulation studies demonstrate the effectiveness and efficiency of the proposed algorithms.",
keywords = "Gradient method, Low Rank, Performance bound, Regression",
author = "Jianhui Chen and Jieping Ye",
year = "2014",
doi = "10.1007/s00180-013-0440-7",
language = "English (US)",
volume = "29",
pages = "623--639",
journal = "Computational Statistics",
issn = "0943-4062",
publisher = "Springer Verlag",
number = "3-4",

}

TY - JOUR

T1 - Sparse trace norm regularization

AU - Chen, Jianhui

AU - Ye, Jieping

PY - 2014

Y1 - 2014

N2 - We study the problem of estimating multiple predictive functions from a dictionary of basis functions in the nonparametric regression setting. Our estimation scheme assumes that each predictive function can be estimated in the form of a linear combination of the basis functions. By assuming that the coefficient matrix admits a sparse low-rank structure, we formulate the function estimation problem as a convex program regularized by the trace norm and the ℓ1-norm simultaneously. We propose to solve the convex program using the accelerated gradient (AG) method; we also develop efficient algorithms to solve the key components in AG. In addition, we conduct theoretical analysis on the proposed function estimation scheme: we derive a key property of the optimal solution to the convex program; based on an assumption on the basis functions, we establish a performance bound of the proposed function estimation scheme (via the composite regularization). Simulation studies demonstrate the effectiveness and efficiency of the proposed algorithms.

AB - We study the problem of estimating multiple predictive functions from a dictionary of basis functions in the nonparametric regression setting. Our estimation scheme assumes that each predictive function can be estimated in the form of a linear combination of the basis functions. By assuming that the coefficient matrix admits a sparse low-rank structure, we formulate the function estimation problem as a convex program regularized by the trace norm and the ℓ1-norm simultaneously. We propose to solve the convex program using the accelerated gradient (AG) method; we also develop efficient algorithms to solve the key components in AG. In addition, we conduct theoretical analysis on the proposed function estimation scheme: we derive a key property of the optimal solution to the convex program; based on an assumption on the basis functions, we establish a performance bound of the proposed function estimation scheme (via the composite regularization). Simulation studies demonstrate the effectiveness and efficiency of the proposed algorithms.

KW - Gradient method

KW - Low Rank

KW - Performance bound

KW - Regression

UR - http://www.scopus.com/inward/record.url?scp=84901847839&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84901847839&partnerID=8YFLogxK

U2 - 10.1007/s00180-013-0440-7

DO - 10.1007/s00180-013-0440-7

M3 - Article

VL - 29

SP - 623

EP - 639

JO - Computational Statistics

JF - Computational Statistics

SN - 0943-4062

IS - 3-4

ER -