Abstract

Information divergence functions allow us to measure distances between probability density functions. We focus on the case where we only have data from the two distributions and have no knowledge of the underlying models from which the data is sampled. In this scenario, we consider an f-divergence for which there exists an asymptotically consistent, nonparametric estimator based on minimum spanning trees, the Dp divergence. Nonparametric estimators are known to have slow convergence rates in higher dimensions (d > 4), resulting in a large bias for small datasets. Based on experimental validation, we conjecture that the original estimator follows a power law convergence model and introduce a new estimator based on a bootstrap sampling scheme that results in a reduced bias. Experiments on real and artificial data show that the new estimator results in improved estimates of the Dp divergence when compared against the original estimator.

Original languageEnglish (US)
Title of host publicationConference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
EditorsMichael B. Matthews
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages875-879
Number of pages5
Volume2017-October
ISBN (Electronic)9781538618233
DOIs
StatePublished - Apr 10 2018
Event51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 - Pacific Grove, United States
Duration: Oct 29 2017Nov 1 2017

Other

Other51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
CountryUnited States
CityPacific Grove
Period10/29/1711/1/17

    Fingerprint

Keywords

  • asymptotic estimator
  • bootstrap estimator
  • minimal graphs
  • nonparametric f-Divergence

ASJC Scopus subject areas

  • Control and Optimization
  • Computer Networks and Communications
  • Hardware and Architecture
  • Signal Processing
  • Biomedical Engineering
  • Instrumentation

Cite this

Kadambi, P., Wisler, A., & Berisha, V. (2018). Improved finite-sample estimate of a nonparametric f-divergence. In M. B. Matthews (Ed.), Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 (Vol. 2017-October, pp. 875-879). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ACSSC.2017.8335474