Attention-based Representation Learning for Time Series with Principal and Residual Space Monitoring

Botao Wang, Fugee Tsung, Hao Yan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The encoder-decoder network is one of the most common deep learning models for time series representation learning and anomaly detection. However, it is hard to reconstruct time series, which is complex, correlated, and lacking in common patterns. In this paper, we apply the attention mechanism to rescale convolution layers and learn representation in the principal and the residual space. To avoid the reconstruction process, we define the residual space by the omitted segments according to the attention score in the encoder. We introduce the temporal information inside the token level and use sparse penalty to improve representation learning. We apply the proposed model to anomaly classification and fault detection experiments on two datasets, i.e. multivariate bearing fault dataset and UCRArchive profile dataset. The result shows that the representation learned by the proposed model is more likely to cluster by category, especially in the residual space. Compared to the baselines and state-of-the-art models, the proposed model has higher accuracy and recall in the limited-labeled situation, which illustrates the stability of the learned representation and its superiority in the downstream tasks.

Original languageEnglish (US)
Title of host publication2022 IEEE 18th International Conference on Automation Science and Engineering, CASE 2022
PublisherIEEE Computer Society
Pages1833-1839
Number of pages7
ISBN (Electronic)9781665490429
DOIs
StatePublished - 2022
Externally publishedYes
Event18th IEEE International Conference on Automation Science and Engineering, CASE 2022 - Mexico City, Mexico
Duration: Aug 20 2022Aug 24 2022

Publication series

NameIEEE International Conference on Automation Science and Engineering
Volume2022-August
ISSN (Print)2161-8070
ISSN (Electronic)2161-8089

Conference

Conference18th IEEE International Conference on Automation Science and Engineering, CASE 2022
Country/TerritoryMexico
CityMexico City
Period8/20/228/24/22

Keywords

  • Anomaly detection
  • attention mechanism
  • representation learning
  • residual space
  • time series

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Attention-based Representation Learning for Time Series with Principal and Residual Space Monitoring'. Together they form a unique fingerprint.

Cite this