A Tunable Loss Function for Binary Classification

Tyler Sypherd, Mario Diaz, Lalitha Sankar, Peter Kairouz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present α-loss, α ϵ [1, ∞], a tunable loss function for binary classification that bridges log-loss (α = 1) and 0-1 loss (α = ∞). We prove that α-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal yet intractable 0-1 loss. For logistic regression-based classification, we provide an upper bound on the difference between the empirical and expected risk for α-loss at the critical points of the empirical risk by exploiting its Lipschitzianity along with recent results on the landscape features of empirical risk functions. Finally, we show that α-loss with α = 2 performs better than log-loss on MNIST for logistic regression.

Original languageEnglish (US)
Title of host publication2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2479-2483
Number of pages5
ISBN (Electronic)9781538692912
DOIs
Publication statusPublished - Jul 2019
Event2019 IEEE International Symposium on Information Theory, ISIT 2019 - Paris, France
Duration: Jul 7 2019Jul 12 2019

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
Volume2019-July
ISSN (Print)2157-8095

Conference

Conference2019 IEEE International Symposium on Information Theory, ISIT 2019
CountryFrance
CityParis
Period7/7/197/12/19

    Fingerprint

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics

Cite this

Sypherd, T., Diaz, M., Sankar, L., & Kairouz, P. (2019). A Tunable Loss Function for Binary Classification. In 2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings (pp. 2479-2483). [8849796] (IEEE International Symposium on Information Theory - Proceedings; Vol. 2019-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT.2019.8849796