Interactive anomaly detection on attributed networks

Kaize Ding, Jundong Li, Huan Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Performing anomaly detection on attributed networks concerns with finding nodes whose patterns or behaviors deviate significantly from the majority of reference nodes. Its success can be easily found in many real-world applications such as network intrusion detection, opinion spam detection and system fault diagnosis, to name a few. Despite their empirical success, a vast majority of existing efforts are overwhelmingly performed in an unsupervised scenario due to the expensive labeling costs of ground truth anomalies. In fact, in many scenarios, a small amount of prior human knowledge of the data is often effortless to obtain, and getting it involved in the learning process has shown to be effective in advancing many important learning tasks. Additionally, since new types of anomalies may constantly arise over time especially in an adversarial environment, the interests of human expert could also change accordingly regarding to the detected anomaly types. It brings further challenges to conventional anomaly detection algorithms as they are often applied in a batch setting and are incapable to interact with the environment. To tackle the above issues, in this paper, we investigate the problem of anomaly detection on attributed networks in an interactive setting by allowing the system to proactively communicate with the human expert in making a limited number of queries about ground truth anomalies. Our objective is to maximize the true anomalies presented to the human expert after a given budget is used up. Along with this line, we formulate the problem through the principled multi-armed bandit framework and develop a novel collaborative contextual bandit algorithm, named GraphUCB. In particular, our developed algorithm: (1) explicitly models the nodal attributes and node dependencies seamlessly in a joint framework; and (2) handles the exploration-exploitation dilemma when querying anomalies of different types. Extensive experiments on real-world datasets show the improvement of the proposed algorithm over the state-of-the-art algorithms.

Original languageEnglish (US)
Title of host publicationWSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining
PublisherAssociation for Computing Machinery, Inc
Pages357-365
Number of pages9
ISBN (Electronic)9781450359405
DOIs
StatePublished - Jan 30 2019
Event12th ACM International Conference on Web Search and Data Mining, WSDM 2019 - Melbourne, Australia
Duration: Feb 11 2019Feb 15 2019

Publication series

NameWSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining

Conference

Conference12th ACM International Conference on Web Search and Data Mining, WSDM 2019
CountryAustralia
CityMelbourne
Period2/11/192/15/19

Fingerprint

Intrusion detection
Labeling
Failure analysis
Costs
Experiments

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software
  • Computer Science Applications

Cite this

Ding, K., Li, J., & Liu, H. (2019). Interactive anomaly detection on attributed networks. In WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining (pp. 357-365). (WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining). Association for Computing Machinery, Inc. https://doi.org/10.1145/3289600.3290964

Interactive anomaly detection on attributed networks. / Ding, Kaize; Li, Jundong; Liu, Huan.

WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc, 2019. p. 357-365 (WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ding, K, Li, J & Liu, H 2019, Interactive anomaly detection on attributed networks. in WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining. WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining, Association for Computing Machinery, Inc, pp. 357-365, 12th ACM International Conference on Web Search and Data Mining, WSDM 2019, Melbourne, Australia, 2/11/19. https://doi.org/10.1145/3289600.3290964
Ding K, Li J, Liu H. Interactive anomaly detection on attributed networks. In WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc. 2019. p. 357-365. (WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining). https://doi.org/10.1145/3289600.3290964
Ding, Kaize ; Li, Jundong ; Liu, Huan. / Interactive anomaly detection on attributed networks. WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc, 2019. pp. 357-365 (WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining).
@inproceedings{252529d62d4b43098342b4971986eb14,
title = "Interactive anomaly detection on attributed networks",
abstract = "Performing anomaly detection on attributed networks concerns with finding nodes whose patterns or behaviors deviate significantly from the majority of reference nodes. Its success can be easily found in many real-world applications such as network intrusion detection, opinion spam detection and system fault diagnosis, to name a few. Despite their empirical success, a vast majority of existing efforts are overwhelmingly performed in an unsupervised scenario due to the expensive labeling costs of ground truth anomalies. In fact, in many scenarios, a small amount of prior human knowledge of the data is often effortless to obtain, and getting it involved in the learning process has shown to be effective in advancing many important learning tasks. Additionally, since new types of anomalies may constantly arise over time especially in an adversarial environment, the interests of human expert could also change accordingly regarding to the detected anomaly types. It brings further challenges to conventional anomaly detection algorithms as they are often applied in a batch setting and are incapable to interact with the environment. To tackle the above issues, in this paper, we investigate the problem of anomaly detection on attributed networks in an interactive setting by allowing the system to proactively communicate with the human expert in making a limited number of queries about ground truth anomalies. Our objective is to maximize the true anomalies presented to the human expert after a given budget is used up. Along with this line, we formulate the problem through the principled multi-armed bandit framework and develop a novel collaborative contextual bandit algorithm, named GraphUCB. In particular, our developed algorithm: (1) explicitly models the nodal attributes and node dependencies seamlessly in a joint framework; and (2) handles the exploration-exploitation dilemma when querying anomalies of different types. Extensive experiments on real-world datasets show the improvement of the proposed algorithm over the state-of-the-art algorithms.",
author = "Kaize Ding and Jundong Li and Huan Liu",
year = "2019",
month = "1",
day = "30",
doi = "10.1145/3289600.3290964",
language = "English (US)",
series = "WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining",
publisher = "Association for Computing Machinery, Inc",
pages = "357--365",
booktitle = "WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining",

}

TY - GEN

T1 - Interactive anomaly detection on attributed networks

AU - Ding, Kaize

AU - Li, Jundong

AU - Liu, Huan

PY - 2019/1/30

Y1 - 2019/1/30

N2 - Performing anomaly detection on attributed networks concerns with finding nodes whose patterns or behaviors deviate significantly from the majority of reference nodes. Its success can be easily found in many real-world applications such as network intrusion detection, opinion spam detection and system fault diagnosis, to name a few. Despite their empirical success, a vast majority of existing efforts are overwhelmingly performed in an unsupervised scenario due to the expensive labeling costs of ground truth anomalies. In fact, in many scenarios, a small amount of prior human knowledge of the data is often effortless to obtain, and getting it involved in the learning process has shown to be effective in advancing many important learning tasks. Additionally, since new types of anomalies may constantly arise over time especially in an adversarial environment, the interests of human expert could also change accordingly regarding to the detected anomaly types. It brings further challenges to conventional anomaly detection algorithms as they are often applied in a batch setting and are incapable to interact with the environment. To tackle the above issues, in this paper, we investigate the problem of anomaly detection on attributed networks in an interactive setting by allowing the system to proactively communicate with the human expert in making a limited number of queries about ground truth anomalies. Our objective is to maximize the true anomalies presented to the human expert after a given budget is used up. Along with this line, we formulate the problem through the principled multi-armed bandit framework and develop a novel collaborative contextual bandit algorithm, named GraphUCB. In particular, our developed algorithm: (1) explicitly models the nodal attributes and node dependencies seamlessly in a joint framework; and (2) handles the exploration-exploitation dilemma when querying anomalies of different types. Extensive experiments on real-world datasets show the improvement of the proposed algorithm over the state-of-the-art algorithms.

AB - Performing anomaly detection on attributed networks concerns with finding nodes whose patterns or behaviors deviate significantly from the majority of reference nodes. Its success can be easily found in many real-world applications such as network intrusion detection, opinion spam detection and system fault diagnosis, to name a few. Despite their empirical success, a vast majority of existing efforts are overwhelmingly performed in an unsupervised scenario due to the expensive labeling costs of ground truth anomalies. In fact, in many scenarios, a small amount of prior human knowledge of the data is often effortless to obtain, and getting it involved in the learning process has shown to be effective in advancing many important learning tasks. Additionally, since new types of anomalies may constantly arise over time especially in an adversarial environment, the interests of human expert could also change accordingly regarding to the detected anomaly types. It brings further challenges to conventional anomaly detection algorithms as they are often applied in a batch setting and are incapable to interact with the environment. To tackle the above issues, in this paper, we investigate the problem of anomaly detection on attributed networks in an interactive setting by allowing the system to proactively communicate with the human expert in making a limited number of queries about ground truth anomalies. Our objective is to maximize the true anomalies presented to the human expert after a given budget is used up. Along with this line, we formulate the problem through the principled multi-armed bandit framework and develop a novel collaborative contextual bandit algorithm, named GraphUCB. In particular, our developed algorithm: (1) explicitly models the nodal attributes and node dependencies seamlessly in a joint framework; and (2) handles the exploration-exploitation dilemma when querying anomalies of different types. Extensive experiments on real-world datasets show the improvement of the proposed algorithm over the state-of-the-art algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85061728794&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061728794&partnerID=8YFLogxK

U2 - 10.1145/3289600.3290964

DO - 10.1145/3289600.3290964

M3 - Conference contribution

T3 - WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining

SP - 357

EP - 365

BT - WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining

PB - Association for Computing Machinery, Inc

ER -