Abstract
For natural language understanding (NLU) technology to be maximally useful, it must be able to process language in a way that is not exclusive to a single task, genre, or dataset. In pursuit of this objective, we introduce the General Language Understanding Evaluation (GLUE) benchmark, a collection of tools for evaluating the performance of models across a diverse set of existing NLU tasks. By including tasks with limited training data, GLUE is designed to favor and encourage models that share general linguistic knowledge across tasks. GLUE also includes a hand-crafted diagnostic test suite that enables detailed linguistic analysis of models. We evaluate baselines based on current methods for transfer and representation learning and find that multi-task training on all tasks performs better than training a separate model per task. However, the low absolute performance of our best model indicates the need for improved general NLU systems.
| Original language | English |
|---|---|
| State | Published - 2019 |
| Externally published | Yes |
| Event | 7th International Conference on Learning Representations, ICLR 2019 - New Orleans, United States Duration: 6 May 2019 → 9 May 2019 |
Conference
| Conference | 7th International Conference on Learning Representations, ICLR 2019 |
|---|---|
| Country/Territory | United States |
| City | New Orleans |
| Period | 6/05/19 → 9/05/19 |