TY - GEN
T1 - Direct Ensemble Estimation of Density Functionals
AU - Wisler, Alan
AU - Moon, Kevin
AU - Berisha, Visar
N1 - Funding Information:
The authors gratefully acknowledge Dennis Wei and Karthikeyan Ramamurthy at IBM Watson Research Labs for their help in discussing the ideas presented in this paper. This research was supported in part by Office of Naval Research grants N000141410722 (Berisha) and N000141712826 (Berisha).
Publisher Copyright:
© 2018 IEEE.
PY - 2018/9/10
Y1 - 2018/9/10
N2 - Estimating density functionals of analog sources is an important problem in statistical signal processing and information theory. Traditionally, estimating these quantities requires either making parametric assumptions about the underlying distributions or using non-parametric density estimation followed by integration. In this paper we introduce a direct nonparametric approach which bypasses the need for density estimation by using the error rates of k-NN classifiers as 'data-driven' basis functions that can be combined to estimate a range of density functionals. However, this method is subject to a non-trivial bias that dramatically slows the rate of convergence in higher dimensions. To overcome this limitation, we develop an ensemble method for estimating the value of the basis function which, under some minor constraints on the smoothness of the underlying distributions, achieves the parametric rate of convergence regardless of data dimension.
AB - Estimating density functionals of analog sources is an important problem in statistical signal processing and information theory. Traditionally, estimating these quantities requires either making parametric assumptions about the underlying distributions or using non-parametric density estimation followed by integration. In this paper we introduce a direct nonparametric approach which bypasses the need for density estimation by using the error rates of k-NN classifiers as 'data-driven' basis functions that can be combined to estimate a range of density functionals. However, this method is subject to a non-trivial bias that dramatically slows the rate of convergence in higher dimensions. To overcome this limitation, we develop an ensemble method for estimating the value of the basis function which, under some minor constraints on the smoothness of the underlying distributions, achieves the parametric rate of convergence regardless of data dimension.
KW - Divergence estimation
KW - Ensemble estimation
KW - K-nearest neighbor classifier
KW - Non-parametric
UR - http://www.scopus.com/inward/record.url?scp=85052215445&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85052215445&partnerID=8YFLogxK
U2 - 10.1109/ICASSP.2018.8462308
DO - 10.1109/ICASSP.2018.8462308
M3 - Conference contribution
AN - SCOPUS:85052215445
SN - 9781538646588
T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
SP - 2866
EP - 2870
BT - 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018
Y2 - 15 April 2018 through 20 April 2018
ER -