TY - GEN
T1 - Self-supervised Novelty Detection for Continual Learning
T2 - 1st International Workshop on Continual Semi-Supervised Learning, CSSL 2021
AU - Sun, Jingbo
AU - Yang, Li
AU - Zhang, Jiaxin
AU - Liu, Frank
AU - Halappanavar, Mahantesh
AU - Fan, Deliang
AU - Cao, Yu
N1 - Funding Information:
Acknowledgements. This research was supported in part by the U.S. Department of Energy, through the Office of Advanced Scientific Computing Research’s “Data-Driven Decision Control for Complex Systems (DnC2S)” project. It was also partially supported by C-BRIC, one of six centers in JUMP, a Semiconductor Research Corporation (SRC) program sponsored by DARPA. Pacific Northwest National Laboratory is operated by Battelle Memorial Institute for the U.S. Department of Energy under Contract No. DE-AC05-76RL01830. Oak Ridge National Laboratory is operated by UT-Battelle LLC for the U.S. Department of Energy under contract number DE-AC05-00OR22725.
Publisher Copyright:
© 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
PY - 2022
Y1 - 2022
N2 - Novelty detection aims to automatically identify out of distribution (OOD) data, without any prior knowledge of them. It is a critical step in continual learning, in order to sense the arrival of new data and initialize the learning process. Conventional methods of OOD detection perform multi-variate analysis on an ensemble of data or features, and usually resort to the supervision with OOD data to improve the accuracy. In reality, such supervision is impractical as one cannot anticipate the anomalous data. In this paper, we propose a novel, self-supervised approach that does not rely on any pre-defined OOD data: (1) The new method evaluates the Mahalanobis distance of the gradients between the in-distribution and OOD data. (2) It is assisted by a self-supervised binary classifier to guide the label selection to generate the gradients, and maximize the Mahalanobis distance. In the evaluation with multiple datasets, such as CIFAR-10, CIFAR-100, SVHN and ImageNet, the proposed approach consistently outperforms state-of-the-art supervised and unsupervised methods in the area under the receiver operating characteristic (AUROC). We further demonstrate that this detector is able to accurately learn one OOD class in continual learning.
AB - Novelty detection aims to automatically identify out of distribution (OOD) data, without any prior knowledge of them. It is a critical step in continual learning, in order to sense the arrival of new data and initialize the learning process. Conventional methods of OOD detection perform multi-variate analysis on an ensemble of data or features, and usually resort to the supervision with OOD data to improve the accuracy. In reality, such supervision is impractical as one cannot anticipate the anomalous data. In this paper, we propose a novel, self-supervised approach that does not rely on any pre-defined OOD data: (1) The new method evaluates the Mahalanobis distance of the gradients between the in-distribution and OOD data. (2) It is assisted by a self-supervised binary classifier to guide the label selection to generate the gradients, and maximize the Mahalanobis distance. In the evaluation with multiple datasets, such as CIFAR-10, CIFAR-100, SVHN and ImageNet, the proposed approach consistently outperforms state-of-the-art supervised and unsupervised methods in the area under the receiver operating characteristic (AUROC). We further demonstrate that this detector is able to accurately learn one OOD class in continual learning.
KW - Continual learning
KW - Mahalanobis distance
KW - Novelty detection
KW - Unsupervised learning
UR - http://www.scopus.com/inward/record.url?scp=85140449254&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85140449254&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-17587-9_9
DO - 10.1007/978-3-031-17587-9_9
M3 - Conference contribution
AN - SCOPUS:85140449254
SN - 9783031175862
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 118
EP - 133
BT - Continual Semi-Supervised Learning - 1st International Workshop, CSSL 2021, Revised Selected Papers
A2 - Cuzzolin, Fabio
A2 - Cannons, Kevin
A2 - Lomonaco, Vincenzo
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 19 August 2021 through 20 August 2021
ER -