Semi-supervised learning with nuclear norm regularization

Fanhua Shang, L. C. Jiao, Yuanyuan Liu, Hanghang Tong

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

Integrating new knowledge sources into various learning tasks to improve their performance has recently become an interesting topic. In this paper we propose a novel semi-supervised learning (SSL) approach, called semi-supervised learning with nuclear norm regularization (SSL-NNR), which can simultaneously handle both sparse labeled data and additional pairwise constraints together with unlabeled data. Specifically, we first construct a unified SSL framework to combine the manifold assumption and the pairwise constraints assumption for classification tasks. Then we provide a modified fixed point continuous algorithm to learn a low-rank kernel matrix that takes advantage of Laplacian spectral regularization. Finally, we develop a two-stage optimization strategy, and present a semi-supervised classification algorithm with enhanced spectral kernel (ESK). Moreover, we also present a theoretical analysis of the proposed ESK algorithm, and derive an easy approach to extend it to out-of-sample data. Experimental results on a variety of synthetic and real-world data sets demonstrate the effectiveness of the proposed ESK algorithm.

Original languageEnglish (US)
Pages (from-to)2323-2336
Number of pages14
JournalPattern Recognition
Volume46
Issue number8
DOIs
StatePublished - Aug 2013
Externally publishedYes

Fingerprint

Supervised learning

Keywords

  • Graph Laplacian
  • Low-rank kernel learning
  • Nuclear norm regularization
  • Pairwise constraints
  • Semi-supervised learning (SSL)

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Signal Processing

Cite this

Semi-supervised learning with nuclear norm regularization. / Shang, Fanhua; Jiao, L. C.; Liu, Yuanyuan; Tong, Hanghang.

In: Pattern Recognition, Vol. 46, No. 8, 08.2013, p. 2323-2336.

Research output: Contribution to journalArticle

Shang, Fanhua ; Jiao, L. C. ; Liu, Yuanyuan ; Tong, Hanghang. / Semi-supervised learning with nuclear norm regularization. In: Pattern Recognition. 2013 ; Vol. 46, No. 8. pp. 2323-2336.
@article{c70157e297bc4ecb89e6fb6a418b8e84,
title = "Semi-supervised learning with nuclear norm regularization",
abstract = "Integrating new knowledge sources into various learning tasks to improve their performance has recently become an interesting topic. In this paper we propose a novel semi-supervised learning (SSL) approach, called semi-supervised learning with nuclear norm regularization (SSL-NNR), which can simultaneously handle both sparse labeled data and additional pairwise constraints together with unlabeled data. Specifically, we first construct a unified SSL framework to combine the manifold assumption and the pairwise constraints assumption for classification tasks. Then we provide a modified fixed point continuous algorithm to learn a low-rank kernel matrix that takes advantage of Laplacian spectral regularization. Finally, we develop a two-stage optimization strategy, and present a semi-supervised classification algorithm with enhanced spectral kernel (ESK). Moreover, we also present a theoretical analysis of the proposed ESK algorithm, and derive an easy approach to extend it to out-of-sample data. Experimental results on a variety of synthetic and real-world data sets demonstrate the effectiveness of the proposed ESK algorithm.",
keywords = "Graph Laplacian, Low-rank kernel learning, Nuclear norm regularization, Pairwise constraints, Semi-supervised learning (SSL)",
author = "Fanhua Shang and Jiao, {L. C.} and Yuanyuan Liu and Hanghang Tong",
year = "2013",
month = "8",
doi = "10.1016/j.patcog.2013.01.009",
language = "English (US)",
volume = "46",
pages = "2323--2336",
journal = "Pattern Recognition",
issn = "0031-3203",
publisher = "Elsevier Limited",
number = "8",

}

TY - JOUR

T1 - Semi-supervised learning with nuclear norm regularization

AU - Shang, Fanhua

AU - Jiao, L. C.

AU - Liu, Yuanyuan

AU - Tong, Hanghang

PY - 2013/8

Y1 - 2013/8

N2 - Integrating new knowledge sources into various learning tasks to improve their performance has recently become an interesting topic. In this paper we propose a novel semi-supervised learning (SSL) approach, called semi-supervised learning with nuclear norm regularization (SSL-NNR), which can simultaneously handle both sparse labeled data and additional pairwise constraints together with unlabeled data. Specifically, we first construct a unified SSL framework to combine the manifold assumption and the pairwise constraints assumption for classification tasks. Then we provide a modified fixed point continuous algorithm to learn a low-rank kernel matrix that takes advantage of Laplacian spectral regularization. Finally, we develop a two-stage optimization strategy, and present a semi-supervised classification algorithm with enhanced spectral kernel (ESK). Moreover, we also present a theoretical analysis of the proposed ESK algorithm, and derive an easy approach to extend it to out-of-sample data. Experimental results on a variety of synthetic and real-world data sets demonstrate the effectiveness of the proposed ESK algorithm.

AB - Integrating new knowledge sources into various learning tasks to improve their performance has recently become an interesting topic. In this paper we propose a novel semi-supervised learning (SSL) approach, called semi-supervised learning with nuclear norm regularization (SSL-NNR), which can simultaneously handle both sparse labeled data and additional pairwise constraints together with unlabeled data. Specifically, we first construct a unified SSL framework to combine the manifold assumption and the pairwise constraints assumption for classification tasks. Then we provide a modified fixed point continuous algorithm to learn a low-rank kernel matrix that takes advantage of Laplacian spectral regularization. Finally, we develop a two-stage optimization strategy, and present a semi-supervised classification algorithm with enhanced spectral kernel (ESK). Moreover, we also present a theoretical analysis of the proposed ESK algorithm, and derive an easy approach to extend it to out-of-sample data. Experimental results on a variety of synthetic and real-world data sets demonstrate the effectiveness of the proposed ESK algorithm.

KW - Graph Laplacian

KW - Low-rank kernel learning

KW - Nuclear norm regularization

KW - Pairwise constraints

KW - Semi-supervised learning (SSL)

UR - http://www.scopus.com/inward/record.url?scp=84875709468&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84875709468&partnerID=8YFLogxK

U2 - 10.1016/j.patcog.2013.01.009

DO - 10.1016/j.patcog.2013.01.009

M3 - Article

VL - 46

SP - 2323

EP - 2336

JO - Pattern Recognition

JF - Pattern Recognition

SN - 0031-3203

IS - 8

ER -