Extend and explain: Interpreting very long language models

Joel Stremmel, Brian L. Hill, Jeffrey Hertzberg, Jaime Murillo, Llewelyn Allotey, Eran Halperin

Research output: Contribution to journalConference articlepeer-review


While Transformer language models (LMs) are state-of-The-Art for information extraction, long text introduces computational challenges requiring suboptimal preprocessing steps or alternative model architectures. Sparse attention LMs can represent longer sequences, overcoming performance hurdles. However, it remains unclear how to explain predictions from these models, as not all tokens attend to each other in the self-Attention layers, and long sequences pose computational challenges for explainability algorithms when runtime depends on document length. These challenges are severe in the medical context where documents can be very long, and machine learning (ML) models must be auditable and trustworthy. We introduce a novel Masked Sampling Procedure (MSP) to identify the text blocks that contribute to a prediction, apply MSP in the context of predicting diagnoses from medical text, and validate our approach with a blind review by two clinicians. Our method identifies 1.7more clinically informative text blocks than the previous state-of-The-Art, runs up to 100faster, and is tractable for generating important phrase pairs. MSP is particularly well-suited to long LMs but can be applied to any text classifier. We provide a general implementation here.

Original languageEnglish
Pages (from-to)218-258
Number of pages41
JournalProceedings of Machine Learning Research
StatePublished - 2022
Externally publishedYes
Event2nd Machine Learning for Health Symposium, ML4H 2022 - Hybrid, New Orleans, United States
Duration: 28 Nov 2022 → …


  • Explainability
  • Interpretability
  • Language Models
  • Transformers


Dive into the research topics of 'Extend and explain: Interpreting very long language models'. Together they form a unique fingerprint.

Cite this