A multitask learning model for online pattern recognition

Seiichi Ozawa, Asim Roy, Dmitri Roussinov

Research output: Contribution to journalArticle

47 Citations (Scopus)

Abstract

This paper presents a new learning algorithm for multitask pattern recognition (MTPR) problems. We consider learning multiple multiclass classification tasks online where no information is ever provided about the task category of a training example. The algorithm thus needs an automated task recognition capability to properly learn the different classification tasks. The learning mode is "online" where training examples for different tasks are mixed in a random fashion and given sequentially one after another. We assume that the classification tasks are related to each other and that both the tasks and their training examples appear in random during "online training." Thus, the learning algorithm has to continually switch from learning one task to another whenever the training examples change to a different task. This also implies that the learning algorithm has to detect task changes automatically and utilize knowledge of previous tasks for learning new tasks fast. The performance of the algorithm is evaluated for ten MTPR problems using five University of California at Irvine (UCI) data sets. The experiments verify that the proposed algorithm can indeed acquire and accumulate task knowledge and that the transfer of knowledge from tasks already learned enhances the speed of knowledge acquisition on new tasks and the final classification accuracy. In addition, the task categorization accuracy is greatly improved for all MTPR problems by introducing the reorganization process even if the presentation order of class training examples is fairly biased.

Original languageEnglish (US)
Pages (from-to)430-445
Number of pages16
JournalIEEE Transactions on Neural Networks
Volume20
Issue number3
DOIs
StatePublished - 2009

Fingerprint

Pattern recognition
Learning algorithms
Knowledge acquisition
Switches
Experiments

Keywords

  • Automated task recognition
  • Knowledge transfer
  • Multitask learning
  • Online learning
  • Pattern recognition

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Software

Cite this

A multitask learning model for online pattern recognition. / Ozawa, Seiichi; Roy, Asim; Roussinov, Dmitri.

In: IEEE Transactions on Neural Networks, Vol. 20, No. 3, 2009, p. 430-445.

Research output: Contribution to journalArticle

Ozawa, Seiichi ; Roy, Asim ; Roussinov, Dmitri. / A multitask learning model for online pattern recognition. In: IEEE Transactions on Neural Networks. 2009 ; Vol. 20, No. 3. pp. 430-445.
@article{d3357c22752849a689516e91340d9fd5,
title = "A multitask learning model for online pattern recognition",
abstract = "This paper presents a new learning algorithm for multitask pattern recognition (MTPR) problems. We consider learning multiple multiclass classification tasks online where no information is ever provided about the task category of a training example. The algorithm thus needs an automated task recognition capability to properly learn the different classification tasks. The learning mode is {"}online{"} where training examples for different tasks are mixed in a random fashion and given sequentially one after another. We assume that the classification tasks are related to each other and that both the tasks and their training examples appear in random during {"}online training.{"} Thus, the learning algorithm has to continually switch from learning one task to another whenever the training examples change to a different task. This also implies that the learning algorithm has to detect task changes automatically and utilize knowledge of previous tasks for learning new tasks fast. The performance of the algorithm is evaluated for ten MTPR problems using five University of California at Irvine (UCI) data sets. The experiments verify that the proposed algorithm can indeed acquire and accumulate task knowledge and that the transfer of knowledge from tasks already learned enhances the speed of knowledge acquisition on new tasks and the final classification accuracy. In addition, the task categorization accuracy is greatly improved for all MTPR problems by introducing the reorganization process even if the presentation order of class training examples is fairly biased.",
keywords = "Automated task recognition, Knowledge transfer, Multitask learning, Online learning, Pattern recognition",
author = "Seiichi Ozawa and Asim Roy and Dmitri Roussinov",
year = "2009",
doi = "10.1109/TNN.2008.2007961",
language = "English (US)",
volume = "20",
pages = "430--445",
journal = "IEEE Transactions on Neural Networks and Learning Systems",
issn = "2162-237X",
publisher = "IEEE Computational Intelligence Society",
number = "3",

}

TY - JOUR

T1 - A multitask learning model for online pattern recognition

AU - Ozawa, Seiichi

AU - Roy, Asim

AU - Roussinov, Dmitri

PY - 2009

Y1 - 2009

N2 - This paper presents a new learning algorithm for multitask pattern recognition (MTPR) problems. We consider learning multiple multiclass classification tasks online where no information is ever provided about the task category of a training example. The algorithm thus needs an automated task recognition capability to properly learn the different classification tasks. The learning mode is "online" where training examples for different tasks are mixed in a random fashion and given sequentially one after another. We assume that the classification tasks are related to each other and that both the tasks and their training examples appear in random during "online training." Thus, the learning algorithm has to continually switch from learning one task to another whenever the training examples change to a different task. This also implies that the learning algorithm has to detect task changes automatically and utilize knowledge of previous tasks for learning new tasks fast. The performance of the algorithm is evaluated for ten MTPR problems using five University of California at Irvine (UCI) data sets. The experiments verify that the proposed algorithm can indeed acquire and accumulate task knowledge and that the transfer of knowledge from tasks already learned enhances the speed of knowledge acquisition on new tasks and the final classification accuracy. In addition, the task categorization accuracy is greatly improved for all MTPR problems by introducing the reorganization process even if the presentation order of class training examples is fairly biased.

AB - This paper presents a new learning algorithm for multitask pattern recognition (MTPR) problems. We consider learning multiple multiclass classification tasks online where no information is ever provided about the task category of a training example. The algorithm thus needs an automated task recognition capability to properly learn the different classification tasks. The learning mode is "online" where training examples for different tasks are mixed in a random fashion and given sequentially one after another. We assume that the classification tasks are related to each other and that both the tasks and their training examples appear in random during "online training." Thus, the learning algorithm has to continually switch from learning one task to another whenever the training examples change to a different task. This also implies that the learning algorithm has to detect task changes automatically and utilize knowledge of previous tasks for learning new tasks fast. The performance of the algorithm is evaluated for ten MTPR problems using five University of California at Irvine (UCI) data sets. The experiments verify that the proposed algorithm can indeed acquire and accumulate task knowledge and that the transfer of knowledge from tasks already learned enhances the speed of knowledge acquisition on new tasks and the final classification accuracy. In addition, the task categorization accuracy is greatly improved for all MTPR problems by introducing the reorganization process even if the presentation order of class training examples is fairly biased.

KW - Automated task recognition

KW - Knowledge transfer

KW - Multitask learning

KW - Online learning

KW - Pattern recognition

UR - http://www.scopus.com/inward/record.url?scp=63049087592&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=63049087592&partnerID=8YFLogxK

U2 - 10.1109/TNN.2008.2007961

DO - 10.1109/TNN.2008.2007961

M3 - Article

C2 - 19188119

AN - SCOPUS:63049087592

VL - 20

SP - 430

EP - 445

JO - IEEE Transactions on Neural Networks and Learning Systems

JF - IEEE Transactions on Neural Networks and Learning Systems

SN - 2162-237X

IS - 3

ER -