Abstract

Estimating density functionals of analog sources is an important problem in statistical signal processing and information theory. Traditionally, estimating these quantities requires either making parametric assumptions about the underlying distributions or using non-parametric density estimation followed by integration. In this paper we introduce a direct nonparametric approach which bypasses the need for density estimation by using the error rates of k-NN classifiers as 'data-driven' basis functions that can be combined to estimate a range of density functionals. However, this method is subject to a non-trivial bias that dramatically slows the rate of convergence in higher dimensions. To overcome this limitation, we develop an ensemble method for estimating the value of the basis function which, under some minor constraints on the smoothness of the underlying distributions, achieves the parametric rate of convergence regardless of data dimension.

Original languageEnglish (US)
Title of host publication2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2866-2870
Number of pages5
Volume2018-April
ISBN (Print)9781538646588
DOIs
StatePublished - Sep 10 2018
Event2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Calgary, Canada
Duration: Apr 15 2018Apr 20 2018

Other

Other2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018
CountryCanada
CityCalgary
Period4/15/184/20/18

Fingerprint

Information theory
Signal processing
Classifiers

Keywords

  • Divergence estimation
  • Ensemble estimation
  • K-nearest neighbor classifier
  • Non-parametric

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Wisler, A., Moon, K., & Berisha, V. (2018). Direct Ensemble Estimation of Density Functionals. In 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings (Vol. 2018-April, pp. 2866-2870). [8462308] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICASSP.2018.8462308

Direct Ensemble Estimation of Density Functionals. / Wisler, Alan; Moon, Kevin; Berisha, Visar.

2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings. Vol. 2018-April Institute of Electrical and Electronics Engineers Inc., 2018. p. 2866-2870 8462308.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wisler, A, Moon, K & Berisha, V 2018, Direct Ensemble Estimation of Density Functionals. in 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings. vol. 2018-April, 8462308, Institute of Electrical and Electronics Engineers Inc., pp. 2866-2870, 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018, Calgary, Canada, 4/15/18. https://doi.org/10.1109/ICASSP.2018.8462308
Wisler A, Moon K, Berisha V. Direct Ensemble Estimation of Density Functionals. In 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings. Vol. 2018-April. Institute of Electrical and Electronics Engineers Inc. 2018. p. 2866-2870. 8462308 https://doi.org/10.1109/ICASSP.2018.8462308
Wisler, Alan ; Moon, Kevin ; Berisha, Visar. / Direct Ensemble Estimation of Density Functionals. 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings. Vol. 2018-April Institute of Electrical and Electronics Engineers Inc., 2018. pp. 2866-2870
@inproceedings{ef18b336d26546fe9c51afcf3d9b6718,
title = "Direct Ensemble Estimation of Density Functionals",
abstract = "Estimating density functionals of analog sources is an important problem in statistical signal processing and information theory. Traditionally, estimating these quantities requires either making parametric assumptions about the underlying distributions or using non-parametric density estimation followed by integration. In this paper we introduce a direct nonparametric approach which bypasses the need for density estimation by using the error rates of k-NN classifiers as 'data-driven' basis functions that can be combined to estimate a range of density functionals. However, this method is subject to a non-trivial bias that dramatically slows the rate of convergence in higher dimensions. To overcome this limitation, we develop an ensemble method for estimating the value of the basis function which, under some minor constraints on the smoothness of the underlying distributions, achieves the parametric rate of convergence regardless of data dimension.",
keywords = "Divergence estimation, Ensemble estimation, K-nearest neighbor classifier, Non-parametric",
author = "Alan Wisler and Kevin Moon and Visar Berisha",
year = "2018",
month = "9",
day = "10",
doi = "10.1109/ICASSP.2018.8462308",
language = "English (US)",
isbn = "9781538646588",
volume = "2018-April",
pages = "2866--2870",
booktitle = "2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Direct Ensemble Estimation of Density Functionals

AU - Wisler, Alan

AU - Moon, Kevin

AU - Berisha, Visar

PY - 2018/9/10

Y1 - 2018/9/10

N2 - Estimating density functionals of analog sources is an important problem in statistical signal processing and information theory. Traditionally, estimating these quantities requires either making parametric assumptions about the underlying distributions or using non-parametric density estimation followed by integration. In this paper we introduce a direct nonparametric approach which bypasses the need for density estimation by using the error rates of k-NN classifiers as 'data-driven' basis functions that can be combined to estimate a range of density functionals. However, this method is subject to a non-trivial bias that dramatically slows the rate of convergence in higher dimensions. To overcome this limitation, we develop an ensemble method for estimating the value of the basis function which, under some minor constraints on the smoothness of the underlying distributions, achieves the parametric rate of convergence regardless of data dimension.

AB - Estimating density functionals of analog sources is an important problem in statistical signal processing and information theory. Traditionally, estimating these quantities requires either making parametric assumptions about the underlying distributions or using non-parametric density estimation followed by integration. In this paper we introduce a direct nonparametric approach which bypasses the need for density estimation by using the error rates of k-NN classifiers as 'data-driven' basis functions that can be combined to estimate a range of density functionals. However, this method is subject to a non-trivial bias that dramatically slows the rate of convergence in higher dimensions. To overcome this limitation, we develop an ensemble method for estimating the value of the basis function which, under some minor constraints on the smoothness of the underlying distributions, achieves the parametric rate of convergence regardless of data dimension.

KW - Divergence estimation

KW - Ensemble estimation

KW - K-nearest neighbor classifier

KW - Non-parametric

UR - http://www.scopus.com/inward/record.url?scp=85052215445&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052215445&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2018.8462308

DO - 10.1109/ICASSP.2018.8462308

M3 - Conference contribution

AN - SCOPUS:85052215445

SN - 9781538646588

VL - 2018-April

SP - 2866

EP - 2870

BT - 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -