Abstract

Information divergence functions allow us to measure distances between probability density functions. We focus on the case where we only have data from the two distributions and have no knowledge of the underlying models from which the data is sampled. In this scenario, we consider an f-divergence for which there exists an asymptotically consistent, nonparametric estimator based on minimum spanning trees, the Dp divergence. Nonparametric estimators are known to have slow convergence rates in higher dimensions (d > 4), resulting in a large bias for small datasets. Based on experimental validation, we conjecture that the original estimator follows a power law convergence model and introduce a new estimator based on a bootstrap sampling scheme that results in a reduced bias. Experiments on real and artificial data show that the new estimator results in improved estimates of the Dp divergence when compared against the original estimator.

Original languageEnglish (US)
Title of host publicationConference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
EditorsMichael B. Matthews
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages875-879
Number of pages5
Volume2017-October
ISBN (Electronic)9781538618233
DOIs
StatePublished - Apr 10 2018
Event51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 - Pacific Grove, United States
Duration: Oct 29 2017Nov 1 2017

Other

Other51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
CountryUnited States
CityPacific Grove
Period10/29/1711/1/17

Fingerprint

F-divergence
estimators
divergence
Estimator
Divergence
Nonparametric Estimator
estimates
Estimate
Probability density function
Sampling
Experimental Validation
Minimum Spanning Tree
Distance Measure
Bootstrap
Higher Dimensions
Convergence Rate
Power Law
Experiments
probability density functions
Scenarios

Keywords

  • asymptotic estimator
  • bootstrap estimator
  • minimal graphs
  • nonparametric f-Divergence

ASJC Scopus subject areas

  • Control and Optimization
  • Computer Networks and Communications
  • Hardware and Architecture
  • Signal Processing
  • Biomedical Engineering
  • Instrumentation

Cite this

Kadambi, P., Wisler, A., & Berisha, V. (2018). Improved finite-sample estimate of a nonparametric f-divergence. In M. B. Matthews (Ed.), Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 (Vol. 2017-October, pp. 875-879). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ACSSC.2017.8335474

Improved finite-sample estimate of a nonparametric f-divergence. / Kadambi, Prad; Wisler, Alan; Berisha, Visar.

Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017. ed. / Michael B. Matthews. Vol. 2017-October Institute of Electrical and Electronics Engineers Inc., 2018. p. 875-879.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kadambi, P, Wisler, A & Berisha, V 2018, Improved finite-sample estimate of a nonparametric f-divergence. in MB Matthews (ed.), Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017. vol. 2017-October, Institute of Electrical and Electronics Engineers Inc., pp. 875-879, 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017, Pacific Grove, United States, 10/29/17. https://doi.org/10.1109/ACSSC.2017.8335474
Kadambi P, Wisler A, Berisha V. Improved finite-sample estimate of a nonparametric f-divergence. In Matthews MB, editor, Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017. Vol. 2017-October. Institute of Electrical and Electronics Engineers Inc. 2018. p. 875-879 https://doi.org/10.1109/ACSSC.2017.8335474
Kadambi, Prad ; Wisler, Alan ; Berisha, Visar. / Improved finite-sample estimate of a nonparametric f-divergence. Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017. editor / Michael B. Matthews. Vol. 2017-October Institute of Electrical and Electronics Engineers Inc., 2018. pp. 875-879
@inproceedings{3338e17fd7ba443397dc10002bc26e44,
title = "Improved finite-sample estimate of a nonparametric f-divergence",
abstract = "Information divergence functions allow us to measure distances between probability density functions. We focus on the case where we only have data from the two distributions and have no knowledge of the underlying models from which the data is sampled. In this scenario, we consider an f-divergence for which there exists an asymptotically consistent, nonparametric estimator based on minimum spanning trees, the Dp divergence. Nonparametric estimators are known to have slow convergence rates in higher dimensions (d > 4), resulting in a large bias for small datasets. Based on experimental validation, we conjecture that the original estimator follows a power law convergence model and introduce a new estimator based on a bootstrap sampling scheme that results in a reduced bias. Experiments on real and artificial data show that the new estimator results in improved estimates of the Dp divergence when compared against the original estimator.",
keywords = "asymptotic estimator, bootstrap estimator, minimal graphs, nonparametric f-Divergence",
author = "Prad Kadambi and Alan Wisler and Visar Berisha",
year = "2018",
month = "4",
day = "10",
doi = "10.1109/ACSSC.2017.8335474",
language = "English (US)",
volume = "2017-October",
pages = "875--879",
editor = "Matthews, {Michael B.}",
booktitle = "Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Improved finite-sample estimate of a nonparametric f-divergence

AU - Kadambi, Prad

AU - Wisler, Alan

AU - Berisha, Visar

PY - 2018/4/10

Y1 - 2018/4/10

N2 - Information divergence functions allow us to measure distances between probability density functions. We focus on the case where we only have data from the two distributions and have no knowledge of the underlying models from which the data is sampled. In this scenario, we consider an f-divergence for which there exists an asymptotically consistent, nonparametric estimator based on minimum spanning trees, the Dp divergence. Nonparametric estimators are known to have slow convergence rates in higher dimensions (d > 4), resulting in a large bias for small datasets. Based on experimental validation, we conjecture that the original estimator follows a power law convergence model and introduce a new estimator based on a bootstrap sampling scheme that results in a reduced bias. Experiments on real and artificial data show that the new estimator results in improved estimates of the Dp divergence when compared against the original estimator.

AB - Information divergence functions allow us to measure distances between probability density functions. We focus on the case where we only have data from the two distributions and have no knowledge of the underlying models from which the data is sampled. In this scenario, we consider an f-divergence for which there exists an asymptotically consistent, nonparametric estimator based on minimum spanning trees, the Dp divergence. Nonparametric estimators are known to have slow convergence rates in higher dimensions (d > 4), resulting in a large bias for small datasets. Based on experimental validation, we conjecture that the original estimator follows a power law convergence model and introduce a new estimator based on a bootstrap sampling scheme that results in a reduced bias. Experiments on real and artificial data show that the new estimator results in improved estimates of the Dp divergence when compared against the original estimator.

KW - asymptotic estimator

KW - bootstrap estimator

KW - minimal graphs

KW - nonparametric f-Divergence

UR - http://www.scopus.com/inward/record.url?scp=85050914751&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050914751&partnerID=8YFLogxK

U2 - 10.1109/ACSSC.2017.8335474

DO - 10.1109/ACSSC.2017.8335474

M3 - Conference contribution

AN - SCOPUS:85050914751

VL - 2017-October

SP - 875

EP - 879

BT - Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017

A2 - Matthews, Michael B.

PB - Institute of Electrical and Electronics Engineers Inc.

ER -