Logistic regression: Tight bounds for stochastic and online optimization

Elad Hazan, Tomer Koren, Kfir Y. Levy

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


The logistic loss function is often advocated in machine learning and statistics as a smooth and strictly convex surrogate for the 0-1 loss. In this paper we investigate the question of whether these smoothness and convexity properties make the logistic loss preferable to other widely considered options such as the hinge loss. We show that in contrast to known asymptotic bounds, as long as the number of prediction/optimization iterations is sub exponential, the logistic loss provides no improvement over a generic non-smooth loss function such as the hinge loss. In particular we show that the convergence rate of stochastic logistic optimization is bounded from below by a polynomial in the diameter of the decision set and the number of prediction iterations, and provide a matching tight upper bound. This resolves the COLT open problem of McMahan and Streeter (2012).

Original languageEnglish
Title of host publicationProceedings of The 27th Conference on Learning Theory
EditorsMaria Florina Balcan, Vitaly Feldman, Csaba Szepesvári
Number of pages13
StatePublished - 2014
Externally publishedYes
Event27th Conference on Learning Theory, COLT 2014 - Barcelona, Spain
Duration: 13 Jun 201415 Jun 2014

Publication series

NameProceedings of Machine Learning Research
ISSN (Print)2640-3498


Conference27th Conference on Learning Theory, COLT 2014


  • Logistic regression
  • Lower bounds
  • Online learning
  • Stochastic optimization


Dive into the research topics of 'Logistic regression: Tight bounds for stochastic and online optimization'. Together they form a unique fingerprint.

Cite this