What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement

Yotam Alexander, Nimrod De La Vega, Noam Razin, Nadav Cohen

Research output: Contribution to journalConference articlepeer-review

Abstract

The question of what makes a data distribution suitable for deep learning is a fundamental open problem.Focusing on locally connected neural networks (a prevalent family of architectures that includes convolutional and recurrent neural networks as well as local self-attention models), we address this problem by adopting theoretical tools from quantum physics.Our main theoretical result states that a certain locally connected neural network is capable of accurate prediction over a data distribution if and only if the data distribution admits low quantum entanglement under certain canonical partitions of features.As a practical application of this result, we derive a preprocessing method for enhancing the suitability of a data distribution to locally connected neural networks.Experiments with widespread models over various datasets demonstrate our findings.We hope that our use of quantum entanglement will encourage further adoption of tools from physics for formally reasoning about the relation between deep learning and real-world data.

Original languageEnglish
JournalAdvances in Neural Information Processing Systems
Volume36
StatePublished - 2023
Event37th Conference on Neural Information Processing Systems, NeurIPS 2023 - New Orleans, United States
Duration: 10 Dec 202316 Dec 2023

Funding

FundersFunder number
Blavatnik Family Foundation
Google
Yandex Initiative in Machine Learning
Adelis Research Fund for Artificial Intelligence
Tel Aviv University
Google Research Gift
Israel Science Foundation1780/21
Israel Science Foundation

    Fingerprint

    Dive into the research topics of 'What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement'. Together they form a unique fingerprint.

    Cite this