Modeling and change detection of dynamic network data by a network state space model

Na Zou, Jing Li

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Dynamic network data are often encountered in social, biological, and engineering domains. There are two types of variability in dynamic network data: variability of natural evolution and variability due to assignable causes. The latter is the “change” referred to in this article. Accurate and timely change detection from dynamic network data is important. However, it has been infrequently studied, with most of the existing research having focused on community detection, prediction, and visualization. Change detection is a classic research area in Statistical Process Control (SPC), and various approaches have been developed for dynamic data in the form of univariate or multivariate time series but not in the form of networks. We propose a Network State Space Model (NSSM) to characterize the natural evolution of dynamic networks. For tractable parameter estimation of the NSSM, we develop an Expectation Propagation algorithm to produce an approximation for the observation equation of the NSSM and then use Expectation-Maximization integrated with Bayesian Optimal Smoothing to estimate the parameters. For change detection, we further propose a Singular Value Decomposition (SVD)-based method that integrates the NSSM with SPC. A realworld application on Enron dynamic email networks is presented, in which our method successfully detects two known changes.

Original languageEnglish (US)
Pages (from-to)45-57
Number of pages13
JournalIISE Transactions
Volume49
Issue number1
DOIs
StatePublished - 2017

Fingerprint

Statistical process control
Electronic mail
Singular value decomposition
Parameter estimation
Time series
Visualization

Keywords

  • Bayesian optimal smoothing
  • Change detection
  • Dynamic network modeling
  • Expectation propagation
  • Statistical process control

ASJC Scopus subject areas

  • Industrial and Manufacturing Engineering

Cite this

Modeling and change detection of dynamic network data by a network state space model. / Zou, Na; Li, Jing.

In: IISE Transactions, Vol. 49, No. 1, 2017, p. 45-57.

Research output: Contribution to journalArticle

@article{68994fc2fc5649d0b2600ad7913fb5f1,
title = "Modeling and change detection of dynamic network data by a network state space model",
abstract = "Dynamic network data are often encountered in social, biological, and engineering domains. There are two types of variability in dynamic network data: variability of natural evolution and variability due to assignable causes. The latter is the “change” referred to in this article. Accurate and timely change detection from dynamic network data is important. However, it has been infrequently studied, with most of the existing research having focused on community detection, prediction, and visualization. Change detection is a classic research area in Statistical Process Control (SPC), and various approaches have been developed for dynamic data in the form of univariate or multivariate time series but not in the form of networks. We propose a Network State Space Model (NSSM) to characterize the natural evolution of dynamic networks. For tractable parameter estimation of the NSSM, we develop an Expectation Propagation algorithm to produce an approximation for the observation equation of the NSSM and then use Expectation-Maximization integrated with Bayesian Optimal Smoothing to estimate the parameters. For change detection, we further propose a Singular Value Decomposition (SVD)-based method that integrates the NSSM with SPC. A realworld application on Enron dynamic email networks is presented, in which our method successfully detects two known changes.",
keywords = "Bayesian optimal smoothing, Change detection, Dynamic network modeling, Expectation propagation, Statistical process control",
author = "Na Zou and Jing Li",
year = "2017",
doi = "10.1080/0740817X.2016.1198065",
language = "English (US)",
volume = "49",
pages = "45--57",
journal = "IISE Transactions",
issn = "2472-5854",
publisher = "Taylor and Francis Ltd.",
number = "1",

}

TY - JOUR

T1 - Modeling and change detection of dynamic network data by a network state space model

AU - Zou, Na

AU - Li, Jing

PY - 2017

Y1 - 2017

N2 - Dynamic network data are often encountered in social, biological, and engineering domains. There are two types of variability in dynamic network data: variability of natural evolution and variability due to assignable causes. The latter is the “change” referred to in this article. Accurate and timely change detection from dynamic network data is important. However, it has been infrequently studied, with most of the existing research having focused on community detection, prediction, and visualization. Change detection is a classic research area in Statistical Process Control (SPC), and various approaches have been developed for dynamic data in the form of univariate or multivariate time series but not in the form of networks. We propose a Network State Space Model (NSSM) to characterize the natural evolution of dynamic networks. For tractable parameter estimation of the NSSM, we develop an Expectation Propagation algorithm to produce an approximation for the observation equation of the NSSM and then use Expectation-Maximization integrated with Bayesian Optimal Smoothing to estimate the parameters. For change detection, we further propose a Singular Value Decomposition (SVD)-based method that integrates the NSSM with SPC. A realworld application on Enron dynamic email networks is presented, in which our method successfully detects two known changes.

AB - Dynamic network data are often encountered in social, biological, and engineering domains. There are two types of variability in dynamic network data: variability of natural evolution and variability due to assignable causes. The latter is the “change” referred to in this article. Accurate and timely change detection from dynamic network data is important. However, it has been infrequently studied, with most of the existing research having focused on community detection, prediction, and visualization. Change detection is a classic research area in Statistical Process Control (SPC), and various approaches have been developed for dynamic data in the form of univariate or multivariate time series but not in the form of networks. We propose a Network State Space Model (NSSM) to characterize the natural evolution of dynamic networks. For tractable parameter estimation of the NSSM, we develop an Expectation Propagation algorithm to produce an approximation for the observation equation of the NSSM and then use Expectation-Maximization integrated with Bayesian Optimal Smoothing to estimate the parameters. For change detection, we further propose a Singular Value Decomposition (SVD)-based method that integrates the NSSM with SPC. A realworld application on Enron dynamic email networks is presented, in which our method successfully detects two known changes.

KW - Bayesian optimal smoothing

KW - Change detection

KW - Dynamic network modeling

KW - Expectation propagation

KW - Statistical process control

UR - http://www.scopus.com/inward/record.url?scp=85018441290&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85018441290&partnerID=8YFLogxK

U2 - 10.1080/0740817X.2016.1198065

DO - 10.1080/0740817X.2016.1198065

M3 - Article

AN - SCOPUS:85018441290

VL - 49

SP - 45

EP - 57

JO - IISE Transactions

JF - IISE Transactions

SN - 2472-5854

IS - 1

ER -