Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps

Oren Barkan, Edan Hauon, Avi Caciularu, Ori Katz, Itzik Malkiel, Omri Armstrong, Noam Koenigstein

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Transformer-based language models significantly advanced the state-of-the-art in many linguistic tasks. As this revolution continues, the ability to explain model predictions has become a major area of interest for the NLP community. In this work, we present Gradient Self-Attention Maps (Grad-SAM) - a novel gradient-based method that analyzes self-attention units and identifies the input elements that explain the model's prediction the best. Extensive evaluations on various benchmarks show that Grad-SAM obtains significant improvements over state-of-the-art alternatives.

Original languageEnglish
Title of host publicationCIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management
Place of PublicationNew York, NY, USA
PublisherAssociation for Computing Machinery
Pages2882-2887
Number of pages6
ISBN (Electronic)9781450384469
ISBN (Print)9781450384469
DOIs
StatePublished - 26 Oct 2021
Event30th ACM International Conference on Information and Knowledge Management, CIKM 2021 - Virtual, Online, Australia
Duration: 1 Nov 20215 Nov 2021

Publication series

NameInternational Conference on Information and Knowledge Management, Proceedings

Conference

Conference30th ACM International Conference on Information and Knowledge Management, CIKM 2021
Country/TerritoryAustralia
CityVirtual, Online
Period1/11/215/11/21

Keywords

  • bert
  • deep learning
  • explainable & interpretable ai
  • nlp
  • self-attention
  • transformers
  • transparent machine learning

Fingerprint

Dive into the research topics of 'Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps'. Together they form a unique fingerprint.

Cite this