A Nonparametric Adaptive Sampling Strategy for Online Monitoring of Big Data Streams

Xiaochen Xian, Andi Wang, Kaibo Liu

Research output: Contribution to journalArticlepeer-review

15 Scopus citations

Abstract

With the rapid advancement of sensor technology, a huge amount of data is generated in various applications, which poses new and unique challenges for statistical process control (SPC). In this article, we propose a nonparametric adaptive sampling (NAS) strategy to online monitor nonnormal big data streams in the context of limited resources, where only a subset of observations are available at each acquisition time. In particular, this proposed method integrates a rank-based CUSUM scheme and an innovative idea that corrects the anti-rank statistics with partial observations, which can effectively detect a wide range of possible mean shifts when data streams are exchangeable and follow arbitrary distributions. Two theoretical properties on the sampling layout of the proposed NAS algorithm are investigated when the process is in control and out of control. Both simulations and case studies are conducted under different scenarios to illustrate and evaluate the performance of the proposed method. Supplementary materials for this article are available online.

Original languageEnglish (US)
Pages (from-to)14-25
Number of pages12
JournalTechnometrics
Volume60
Issue number1
DOIs
StatePublished - Jan 2 2018
Externally publishedYes

Keywords

  • Distribution-free
  • Multivariate CUSUM procedure
  • Partial observations
  • Process change detection
  • Statistical process control

ASJC Scopus subject areas

  • Statistics and Probability
  • Modeling and Simulation
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'A Nonparametric Adaptive Sampling Strategy for Online Monitoring of Big Data Streams'. Together they form a unique fingerprint.

Cite this