Multiclass classification by sparse multinomial logistic regression

Felix Abramovich*, Vadim Grinshtein, Tomer Levy

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

17 Scopus citations


In this paper we consider high-dimensional multiclass classification by sparse multinomial logistic regression. We propose first a feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the nonasymptotic bounds for misclassification excess risk of the resulting classifier. We establish also their tightness by deriving the corresponding minimax lower bounds. In particular, we show that there is a phase transition between small and large number of classes. The bounds can be reduced under the additional low noise condition. To find a penalized maximum likelihood solution with a complexity penalty requires, however, a combinatorial search over all possible models. To design a feature selection procedure computationally feasible for high-dimensional data, we propose multinomial logistic group Lasso and Slope classifiers and show that they also achieve the minimax order.

Original languageEnglish
Article number9410597
Pages (from-to)4637-4646
Number of pages10
JournalIEEE Transactions on Information Theory
Issue number7
StatePublished - Jul 2021


FundersFunder number
Israel Science FoundationISF-589/18


    • Complexity penalty
    • Convex relaxation
    • Feature selection
    • High-dimensionality
    • Minimaxity
    • Misclassification excess risk
    • Sparsity


    Dive into the research topics of 'Multiclass classification by sparse multinomial logistic regression'. Together they form a unique fingerprint.

    Cite this