Tensor-tensor algebra for optimal representation and compression of multiway data

Misha E. Kilmer, Lior Horesh, Haim Avron, Elizabeth Newman*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

With the advent of machine learning and its overarching pervasiveness it is imperative to devise ways to represent large datasets efficiently while distilling intrinsic features necessary for subsequent analysis. The primary workhorse used in data dimensionality reduction and feature extraction has been the matrix singular value decomposition (SVD), which presupposes that data have been arranged in matrix format. A primary goal in this study is to show that high-dimensional datasets are more compressible when treated as tensors (i.e., multiway arrays) and compressed via tensor-SVDs under the tensor-tensor product constructs and its generalizations. We begin by proving Eckart–Young optimality results for families of tensor-SVDs under two different truncation strategies. Since such optimality properties can be proven in both matrix and tensor-based algebras, a fundamental question arises: Does the tensor construct subsume the matrix construct in terms of representation efficiency? The answer is positive, as proven by showing that a tensor-tensor representation of an equal dimensional spanning space can be superior to its matrix counterpart. We then use these optimality results to investigate how the compressed representation provided by the truncated tensor SVD is related both theoretically and empirically to its two closest tensor-based analogs, the truncated high-order SVD and the truncated tensor-train SVD.

Original languageEnglish
Article numberPNAS 2021 Vol. 118 No. 28 e2015851118
JournalProceedings of the National Academy of Sciences of the United States of America
Volume118
Issue number28
DOIs
StatePublished - 13 Jul 2021

Funding

FundersFunder number
IBM Thomas J. Watson Research Center
Tufts T-Tripods Institute
National Science FoundationCCF-1934553, DMS-1821148

    Keywords

    • Compression
    • Multiway data
    • Rank
    • SVD
    • Tensor

    Fingerprint

    Dive into the research topics of 'Tensor-tensor algebra for optimal representation and compression of multiway data'. Together they form a unique fingerprint.

    Cite this