Self-Supervised Transformers for fMRI representation

Itzik Malkiel; Gony Rosenman; Lior Wolf; Talma Hendler

Self-Supervised Transformers for fMRI representation

Itzik Malkiel
, Gony Rosenman
, Lior Wolf
, Talma Hendler

Research output: Contribution to journal › Conference article › peer-review

23 Scopus citations

Abstract

We present TFF, which is a Transformer framework for the analysis of functional Magnetic Resonance Imaging (fMRI) data. TFF employs a two-phase training approach. First, self-supervised training is applied to a collection of fMRI scans, where the model is trained to reconstruct 3D volume data. Second, the pre-trained model is fine-tuned on specific tasks, utilizing ground truth labels. Our results show state-of-the-art performance on a variety of fMRI tasks, including age and gender prediction, as well as schizophrenia recognition. Our code for the training, network architecture, and results is attached as supplementary material.

Original language	English
Pages (from-to)	895-913
Number of pages	19
Journal	Proceedings of Machine Learning Research
Volume	172
State	Published - 2022
Event	5th International Conference on Medical Imaging with Deep Learning, MIDL 2022 - Zurich, Switzerland Duration: 6 Jul 2022 → 8 Jul 2022

Funding

Funders	Funder number
Horizon 2020 Framework Programme	ERC CoG 725974
European Research Council
Israel Science Foundation	2923/20

Keywords

Self-supervision
Transformers
fMRI

Cite this

@article{b14375b562c144d49cf641ef17607ef3,

title = "Self-Supervised Transformers for fMRI representation",

abstract = "We present TFF, which is a Transformer framework for the analysis of functional Magnetic Resonance Imaging (fMRI) data. TFF employs a two-phase training approach. First, self-supervised training is applied to a collection of fMRI scans, where the model is trained to reconstruct 3D volume data. Second, the pre-trained model is fine-tuned on specific tasks, utilizing ground truth labels. Our results show state-of-the-art performance on a variety of fMRI tasks, including age and gender prediction, as well as schizophrenia recognition. Our code for the training, network architecture, and results is attached as supplementary material.",

keywords = "Self-supervision, Transformers, fMRI",

author = "Itzik Malkiel and Gony Rosenman and Lior Wolf and Talma Hendler",

note = "Publisher Copyright: {\textcopyright} 2022 Proceedings of Machine Learning Research. All rights reserved.; 5th International Conference on Medical Imaging with Deep Learning, MIDL 2022 ; Conference date: 06-07-2022 Through 08-07-2022",

year = "2022",

language = "אנגלית",

volume = "172",

pages = "895--913",

journal = "Proceedings of Machine Learning Research",

issn = "2640-3498",

publisher = "ML Research Press",

}

TY - JOUR

T1 - Self-Supervised Transformers for fMRI representation

AU - Malkiel, Itzik

AU - Rosenman, Gony

AU - Wolf, Lior

AU - Hendler, Talma

PY - 2022

Y1 - 2022

N2 - We present TFF, which is a Transformer framework for the analysis of functional Magnetic Resonance Imaging (fMRI) data. TFF employs a two-phase training approach. First, self-supervised training is applied to a collection of fMRI scans, where the model is trained to reconstruct 3D volume data. Second, the pre-trained model is fine-tuned on specific tasks, utilizing ground truth labels. Our results show state-of-the-art performance on a variety of fMRI tasks, including age and gender prediction, as well as schizophrenia recognition. Our code for the training, network architecture, and results is attached as supplementary material.

AB - We present TFF, which is a Transformer framework for the analysis of functional Magnetic Resonance Imaging (fMRI) data. TFF employs a two-phase training approach. First, self-supervised training is applied to a collection of fMRI scans, where the model is trained to reconstruct 3D volume data. Second, the pre-trained model is fine-tuned on specific tasks, utilizing ground truth labels. Our results show state-of-the-art performance on a variety of fMRI tasks, including age and gender prediction, as well as schizophrenia recognition. Our code for the training, network architecture, and results is attached as supplementary material.

KW - Self-supervision

KW - Transformers

KW - fMRI

UR - https://www.scopus.com/pages/publications/85175423715

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.conferencearticle???

AN - SCOPUS:85175423715

SN - 2640-3498

VL - 172

SP - 895

EP - 913

JO - Proceedings of Machine Learning Research

JF - Proceedings of Machine Learning Research

T2 - 5th International Conference on Medical Imaging with Deep Learning, MIDL 2022

Y2 - 6 July 2022 through 8 July 2022

ER -

Self-Supervised Transformers for fMRI representation

Abstract

Funding

Keywords

Other files and links

Fingerprint

Cite this