Self-supervised Novelty Detection for Continual Learning: A Gradient-Based Approach Boosted by Binary Classification

Jingbo Sun, Li Yang, Jiaxin Zhang, Frank Liu, Mahantesh Halappanavar, Deliang Fan, Yu Cao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Novelty detection aims to automatically identify out of distribution (OOD) data, without any prior knowledge of them. It is a critical step in continual learning, in order to sense the arrival of new data and initialize the learning process. Conventional methods of OOD detection perform multi-variate analysis on an ensemble of data or features, and usually resort to the supervision with OOD data to improve the accuracy. In reality, such supervision is impractical as one cannot anticipate the anomalous data. In this paper, we propose a novel, self-supervised approach that does not rely on any pre-defined OOD data: (1) The new method evaluates the Mahalanobis distance of the gradients between the in-distribution and OOD data. (2) It is assisted by a self-supervised binary classifier to guide the label selection to generate the gradients, and maximize the Mahalanobis distance. In the evaluation with multiple datasets, such as CIFAR-10, CIFAR-100, SVHN and ImageNet, the proposed approach consistently outperforms state-of-the-art supervised and unsupervised methods in the area under the receiver operating characteristic (AUROC). We further demonstrate that this detector is able to accurately learn one OOD class in continual learning.

Original languageEnglish (US)
Title of host publicationContinual Semi-Supervised Learning - 1st International Workshop, CSSL 2021, Revised Selected Papers
EditorsFabio Cuzzolin, Kevin Cannons, Vincenzo Lomonaco
PublisherSpringer Science and Business Media Deutschland GmbH
Pages118-133
Number of pages16
ISBN (Print)9783031175862
DOIs
StatePublished - 2022
Externally publishedYes
Event1st International Workshop on Continual Semi-Supervised Learning, CSSL 2021 - Virtual, Online
Duration: Aug 19 2021Aug 20 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13418 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference1st International Workshop on Continual Semi-Supervised Learning, CSSL 2021
CityVirtual, Online
Period8/19/218/20/21

Keywords

  • Continual learning
  • Mahalanobis distance
  • Novelty detection
  • Unsupervised learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Self-supervised Novelty Detection for Continual Learning: A Gradient-Based Approach Boosted by Binary Classification'. Together they form a unique fingerprint.

Cite this