Can Latent Alignments Improve Autoregressive Machine Translation?

Adi Haviv, Lior Vassertail, Omer Levy

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Latent alignment objectives such as CTC and AXE significantly improve non-autoregressive machine translation models. Can they improve autoregressive models as well? We explore the possibility of training autoregressive machine translation models with latent alignment objectives, and observe that, in practice, this approach results in degenerate models. We provide a theoretical explanation for these empirical results, and prove that latent alignment objectives are incompatible with teacher forcing.
Original languageEnglish
Title of host publicationProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
EditorsKristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
PublisherAssociation for Computational Linguistics
Pages2637-2641
Number of pages5
ISBN (Electronic)978-1-954085-46-6
DOIs
StatePublished - 1 Jun 2021
Event2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Virtual
Duration: 6 Jun 202111 Jun 2021

Conference

Conference2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Abbreviated titleNAACL 2021
Period6/06/2111/06/21

Fingerprint

Dive into the research topics of 'Can Latent Alignments Improve Autoregressive Machine Translation?'. Together they form a unique fingerprint.

Cite this