Exponentiated gradient algorithms for log-linear structured prediction

Amir Globerson, Terry Y. Koo, Xavier Carreras, Michael Collins

Research output: Contribution to conferencePaperpeer-review

Abstract

Conditional log-linear models are a commonly used method for structured prediction. Efficient learning of parameters in these models is therefore an important problem. This paper describes an exponentiated gradient (EG) algorithm for training such models. EG is applied to the convex dual of the maximum likelihood objective; this results in both sequential and parallel update algorithms, where in the sequential algorithm parameters are updated in an online fashion. We provide a convergence proof for both algorithms. Our analysis also simplifies previous results on EG for max-margin models, and leads to a tighter bound on convergence rates. Experiments on a large-scale parsing task show that the proposed algorithm converges much faster than conjugate-gradient and L-BFGS approaches both in terms of optimization objective and test error.

Original languageEnglish
Pages305-312
Number of pages8
DOIs
StatePublished - 2007
Externally publishedYes
Event24th International Conference on Machine Learning, ICML 2007 - Corvalis, OR, United States
Duration: 20 Jun 200724 Jun 2007

Conference

Conference24th International Conference on Machine Learning, ICML 2007
Country/TerritoryUnited States
CityCorvalis, OR
Period20/06/0724/06/07

Fingerprint

Dive into the research topics of 'Exponentiated gradient algorithms for log-linear structured prediction'. Together they form a unique fingerprint.

Cite this