TY - GEN
T1 - A Tunable Loss Function for Binary Classification
AU - Sypherd, Tyler
AU - Diaz, Mario
AU - Sankar, Lalitha
AU - Kairouz, Peter
N1 - Funding Information:
This material is based upon work supported by the National Science Foundation under Grant Nos. CCF-1350914 and CIF-1815261.
Publisher Copyright:
© 2019 IEEE.
PY - 2019/7
Y1 - 2019/7
N2 - We present α-loss, α ϵ [1, ∞], a tunable loss function for binary classification that bridges log-loss (α = 1) and 0-1 loss (α = ∞). We prove that α-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal yet intractable 0-1 loss. For logistic regression-based classification, we provide an upper bound on the difference between the empirical and expected risk for α-loss at the critical points of the empirical risk by exploiting its Lipschitzianity along with recent results on the landscape features of empirical risk functions. Finally, we show that α-loss with α = 2 performs better than log-loss on MNIST for logistic regression.
AB - We present α-loss, α ϵ [1, ∞], a tunable loss function for binary classification that bridges log-loss (α = 1) and 0-1 loss (α = ∞). We prove that α-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal yet intractable 0-1 loss. For logistic regression-based classification, we provide an upper bound on the difference between the empirical and expected risk for α-loss at the critical points of the empirical risk by exploiting its Lipschitzianity along with recent results on the landscape features of empirical risk functions. Finally, we show that α-loss with α = 2 performs better than log-loss on MNIST for logistic regression.
UR - http://www.scopus.com/inward/record.url?scp=85073160283&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85073160283&partnerID=8YFLogxK
U2 - 10.1109/ISIT.2019.8849796
DO - 10.1109/ISIT.2019.8849796
M3 - Conference contribution
AN - SCOPUS:85073160283
T3 - IEEE International Symposium on Information Theory - Proceedings
SP - 2479
EP - 2483
BT - 2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2019 IEEE International Symposium on Information Theory, ISIT 2019
Y2 - 7 July 2019 through 12 July 2019
ER -