Sketched covariance testing: A compression-statistics tradeoff

Gautam Dasarathy; Parikshit Shah; Richard G. Baraniuk

doi:10.1109/ISIT.2017.8006933

Sketched covariance testing: A compression-statistics tradeoff

Gautam Dasarathy, Parikshit Shah, Richard G. Baraniuk

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Scopus citations

Abstract

Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix Σ0, the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a 'sketching' matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.

Original language	English (US)
Title of host publication	2017 IEEE International Symposium on Information Theory, ISIT 2017
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2268-2272
Number of pages	5
ISBN (Electronic)	9781509040964
DOIs	https://doi.org/10.1109/ISIT.2017.8006933
State	Published - Aug 9 2017
Externally published	Yes
Event	2017 IEEE International Symposium on Information Theory, ISIT 2017 - Aachen, Germany Duration: Jun 25 2017 → Jun 30 2017

Publication series

Name	IEEE International Symposium on Information Theory - Proceedings
ISSN (Print)	2157-8095

Other

Other	2017 IEEE International Symposium on Information Theory, ISIT 2017
Country/Territory	Germany
City	Aachen
Period	6/25/17 → 6/30/17

ASJC Scopus subject areas

Theoretical Computer Science
Information Systems
Modeling and Simulation
Applied Mathematics

Access to Document

10.1109/ISIT.2017.8006933

Cite this

Dasarathy, G., Shah, P., & Baraniuk, R. G. (2017). Sketched covariance testing: A compression-statistics tradeoff. In 2017 IEEE International Symposium on Information Theory, ISIT 2017 (pp. 2268-2272). Article 8006933 (IEEE International Symposium on Information Theory - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT.2017.8006933

Sketched covariance testing: A compression-statistics tradeoff. / Dasarathy, Gautam; Shah, Parikshit; Baraniuk, Richard G.
2017 IEEE International Symposium on Information Theory, ISIT 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 2268-2272 8006933 (IEEE International Symposium on Information Theory - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Dasarathy, G, Shah, P & Baraniuk, RG 2017, Sketched covariance testing: A compression-statistics tradeoff. in 2017 IEEE International Symposium on Information Theory, ISIT 2017., 8006933, IEEE International Symposium on Information Theory - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 2268-2272, 2017 IEEE International Symposium on Information Theory, ISIT 2017, Aachen, Germany, 6/25/17. https://doi.org/10.1109/ISIT.2017.8006933

@inproceedings{704790a128b345eb8445f70994e5a85c,

title = "Sketched covariance testing: A compression-statistics tradeoff",

abstract = "Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix Σ0, the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a 'sketching' matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.",

author = "Gautam Dasarathy and Parikshit Shah and Baraniuk, {Richard G.}",

year = "2017",

month = aug,

day = "9",

doi = "10.1109/ISIT.2017.8006933",

language = "English (US)",

series = "IEEE International Symposium on Information Theory - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2268--2272",

booktitle = "2017 IEEE International Symposium on Information Theory, ISIT 2017",

note = "2017 IEEE International Symposium on Information Theory, ISIT 2017 ; Conference date: 25-06-2017 Through 30-06-2017",

}

TY - GEN

T1 - Sketched covariance testing

T2 - 2017 IEEE International Symposium on Information Theory, ISIT 2017

AU - Dasarathy, Gautam

AU - Shah, Parikshit

AU - Baraniuk, Richard G.

PY - 2017/8/9

Y1 - 2017/8/9

N2 - Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix Σ0, the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a 'sketching' matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.

AB - Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix Σ0, the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a 'sketching' matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.

UR - http://www.scopus.com/inward/record.url?scp=85034039775&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85034039775&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2017.8006933

DO - 10.1109/ISIT.2017.8006933

M3 - Conference contribution

AN - SCOPUS:85034039775

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 2268

EP - 2272

BT - 2017 IEEE International Symposium on Information Theory, ISIT 2017

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 25 June 2017 through 30 June 2017

ER -

Sketched covariance testing: A compression-statistics tradeoff

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this