Abstract

Nowadays, temporal data is generated at an unprecedentedspeed from a variety of applications, such as wearable devices, sensor networks, wireless networks, etc. In contrast to suchlarge amount of temporal data, it is usually the case that onlya small portion of them contains information of interest. Forexample, for the ECG signals collected by wearable devices, most of them collected from healthy people are normal, andonly a small number of them collected from people with certain heart diseases are abnormal. Furthermore, even forthe abnormal temporal sequences, the abnormal patterns mayonly be present in a few time segments and are similar amongthemselves, forming a rare category of temporal patterns. Forexample, the ECG signal collected from an individual with acertain heart disease may be normal in most time segments, and abnormal in only a few time segments, exhibiting similarpatterns. What is even more challenging is that such raretemporal patterns are often non-separable from the normalones. Existing works on outlier detection for temporal datafocus on detecting either the abnormal sequences as a whole, orthe abnormal time segments directly, ignoring the relationshipbetween abnormal sequences and abnormal time segments.Moreover, the abnormal patterns are typically treated asisolated outliers instead of a rare category with self-similarity. In this paper, for the first time, we propose a bi-level(sequence-level/ segment-level) model for rare temporal patterndetection. It is based on an optimization frameworkthat fully exploits the bi-level structure in the data, i.e., therelationship between abnormal sequences and abnormal timesegments. Furthermore, it uses sequence-specific simple hiddenMarkov models to obtain segment-level labels, and leverages the similarity among abnormal time segments to estimate the model parameters. To solve the optimization framework, we propose the unsupervised algorithm BIRAD, and also thesemi-supervised version BIRAD-K which learns from a single labeled example. Experimental results on both synthetic andreal data sets demonstrate the performance of the proposedalgorithms from multiple aspects, outperforming state-of-The-Arttechniques on both temporal outlier detection and rarecategory analysis.

Original languageEnglish (US)
Title of host publicationProceedings - 16th IEEE International Conference on Data Mining, ICDM 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages719-728
Number of pages10
ISBN (Electronic)9781509054725
DOIs
StatePublished - Jan 31 2017
Event16th IEEE International Conference on Data Mining, ICDM 2016 - Barcelona, Catalonia, Spain
Duration: Dec 12 2016Dec 15 2016

Other

Other16th IEEE International Conference on Data Mining, ICDM 2016
CountrySpain
CityBarcelona, Catalonia
Period12/12/1612/15/16

Fingerprint

Electrocardiography
Sensor networks
Labels
Wireless networks

Keywords

  • Rare category detection
  • Temporal data mining
  • Time segments
  • Time series

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Zhou, D., He, J., Cao, Y., & Seo, J. (2017). Bi-Level rare temporal pattern detection. In Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016 (pp. 719-728). [7837896] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDM.2016.16

Bi-Level rare temporal pattern detection. / Zhou, Dawei; He, Jingrui; Cao, Yu; Seo, Jae-sun.

Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016. Institute of Electrical and Electronics Engineers Inc., 2017. p. 719-728 7837896.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhou, D, He, J, Cao, Y & Seo, J 2017, Bi-Level rare temporal pattern detection. in Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016., 7837896, Institute of Electrical and Electronics Engineers Inc., pp. 719-728, 16th IEEE International Conference on Data Mining, ICDM 2016, Barcelona, Catalonia, Spain, 12/12/16. https://doi.org/10.1109/ICDM.2016.16
Zhou D, He J, Cao Y, Seo J. Bi-Level rare temporal pattern detection. In Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016. Institute of Electrical and Electronics Engineers Inc. 2017. p. 719-728. 7837896 https://doi.org/10.1109/ICDM.2016.16
Zhou, Dawei ; He, Jingrui ; Cao, Yu ; Seo, Jae-sun. / Bi-Level rare temporal pattern detection. Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 719-728
@inproceedings{7e458fb9fada46e5993c33661a8c25aa,
title = "Bi-Level rare temporal pattern detection",
abstract = "Nowadays, temporal data is generated at an unprecedentedspeed from a variety of applications, such as wearable devices, sensor networks, wireless networks, etc. In contrast to suchlarge amount of temporal data, it is usually the case that onlya small portion of them contains information of interest. Forexample, for the ECG signals collected by wearable devices, most of them collected from healthy people are normal, andonly a small number of them collected from people with certain heart diseases are abnormal. Furthermore, even forthe abnormal temporal sequences, the abnormal patterns mayonly be present in a few time segments and are similar amongthemselves, forming a rare category of temporal patterns. Forexample, the ECG signal collected from an individual with acertain heart disease may be normal in most time segments, and abnormal in only a few time segments, exhibiting similarpatterns. What is even more challenging is that such raretemporal patterns are often non-separable from the normalones. Existing works on outlier detection for temporal datafocus on detecting either the abnormal sequences as a whole, orthe abnormal time segments directly, ignoring the relationshipbetween abnormal sequences and abnormal time segments.Moreover, the abnormal patterns are typically treated asisolated outliers instead of a rare category with self-similarity. In this paper, for the first time, we propose a bi-level(sequence-level/ segment-level) model for rare temporal patterndetection. It is based on an optimization frameworkthat fully exploits the bi-level structure in the data, i.e., therelationship between abnormal sequences and abnormal timesegments. Furthermore, it uses sequence-specific simple hiddenMarkov models to obtain segment-level labels, and leverages the similarity among abnormal time segments to estimate the model parameters. To solve the optimization framework, we propose the unsupervised algorithm BIRAD, and also thesemi-supervised version BIRAD-K which learns from a single labeled example. Experimental results on both synthetic andreal data sets demonstrate the performance of the proposedalgorithms from multiple aspects, outperforming state-of-The-Arttechniques on both temporal outlier detection and rarecategory analysis.",
keywords = "Rare category detection, Temporal data mining, Time segments, Time series",
author = "Dawei Zhou and Jingrui He and Yu Cao and Jae-sun Seo",
year = "2017",
month = "1",
day = "31",
doi = "10.1109/ICDM.2016.16",
language = "English (US)",
pages = "719--728",
booktitle = "Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Bi-Level rare temporal pattern detection

AU - Zhou, Dawei

AU - He, Jingrui

AU - Cao, Yu

AU - Seo, Jae-sun

PY - 2017/1/31

Y1 - 2017/1/31

N2 - Nowadays, temporal data is generated at an unprecedentedspeed from a variety of applications, such as wearable devices, sensor networks, wireless networks, etc. In contrast to suchlarge amount of temporal data, it is usually the case that onlya small portion of them contains information of interest. Forexample, for the ECG signals collected by wearable devices, most of them collected from healthy people are normal, andonly a small number of them collected from people with certain heart diseases are abnormal. Furthermore, even forthe abnormal temporal sequences, the abnormal patterns mayonly be present in a few time segments and are similar amongthemselves, forming a rare category of temporal patterns. Forexample, the ECG signal collected from an individual with acertain heart disease may be normal in most time segments, and abnormal in only a few time segments, exhibiting similarpatterns. What is even more challenging is that such raretemporal patterns are often non-separable from the normalones. Existing works on outlier detection for temporal datafocus on detecting either the abnormal sequences as a whole, orthe abnormal time segments directly, ignoring the relationshipbetween abnormal sequences and abnormal time segments.Moreover, the abnormal patterns are typically treated asisolated outliers instead of a rare category with self-similarity. In this paper, for the first time, we propose a bi-level(sequence-level/ segment-level) model for rare temporal patterndetection. It is based on an optimization frameworkthat fully exploits the bi-level structure in the data, i.e., therelationship between abnormal sequences and abnormal timesegments. Furthermore, it uses sequence-specific simple hiddenMarkov models to obtain segment-level labels, and leverages the similarity among abnormal time segments to estimate the model parameters. To solve the optimization framework, we propose the unsupervised algorithm BIRAD, and also thesemi-supervised version BIRAD-K which learns from a single labeled example. Experimental results on both synthetic andreal data sets demonstrate the performance of the proposedalgorithms from multiple aspects, outperforming state-of-The-Arttechniques on both temporal outlier detection and rarecategory analysis.

AB - Nowadays, temporal data is generated at an unprecedentedspeed from a variety of applications, such as wearable devices, sensor networks, wireless networks, etc. In contrast to suchlarge amount of temporal data, it is usually the case that onlya small portion of them contains information of interest. Forexample, for the ECG signals collected by wearable devices, most of them collected from healthy people are normal, andonly a small number of them collected from people with certain heart diseases are abnormal. Furthermore, even forthe abnormal temporal sequences, the abnormal patterns mayonly be present in a few time segments and are similar amongthemselves, forming a rare category of temporal patterns. Forexample, the ECG signal collected from an individual with acertain heart disease may be normal in most time segments, and abnormal in only a few time segments, exhibiting similarpatterns. What is even more challenging is that such raretemporal patterns are often non-separable from the normalones. Existing works on outlier detection for temporal datafocus on detecting either the abnormal sequences as a whole, orthe abnormal time segments directly, ignoring the relationshipbetween abnormal sequences and abnormal time segments.Moreover, the abnormal patterns are typically treated asisolated outliers instead of a rare category with self-similarity. In this paper, for the first time, we propose a bi-level(sequence-level/ segment-level) model for rare temporal patterndetection. It is based on an optimization frameworkthat fully exploits the bi-level structure in the data, i.e., therelationship between abnormal sequences and abnormal timesegments. Furthermore, it uses sequence-specific simple hiddenMarkov models to obtain segment-level labels, and leverages the similarity among abnormal time segments to estimate the model parameters. To solve the optimization framework, we propose the unsupervised algorithm BIRAD, and also thesemi-supervised version BIRAD-K which learns from a single labeled example. Experimental results on both synthetic andreal data sets demonstrate the performance of the proposedalgorithms from multiple aspects, outperforming state-of-The-Arttechniques on both temporal outlier detection and rarecategory analysis.

KW - Rare category detection

KW - Temporal data mining

KW - Time segments

KW - Time series

UR - http://www.scopus.com/inward/record.url?scp=85014566739&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85014566739&partnerID=8YFLogxK

U2 - 10.1109/ICDM.2016.16

DO - 10.1109/ICDM.2016.16

M3 - Conference contribution

SP - 719

EP - 728

BT - Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -