A function space analysis of finite neural networks with insights from sampling theory

Research output: Contribution to journalArticlepeer-review

Abstract

This work suggests using sampling theory to analyze the function space represented by interpolating mappings. While the analysis in this paper is general, we focus it on neural networks with bounded weights that are known with their ability to interpolate (fit) the training data. First, we show, under the assumption of a finite input domain, which is the common case in training neural networks, that the function space generated by multi-layer networks with bounded weights, and non-expansive activation functions are smooth. This extends over previous works that show results for the case of infinite width ReLU networks. Then, under the assumption that the input is band-limited, we provide novel error bounds for univariate neural networks. We analyze both deterministic uniform and random sampling showing the advantage of the former.

Keywords

  • Discrete Fourier transforms
  • Interpolation
  • Neural networks
  • Splines (mathematics)
  • Training
  • Training data
  • Transforms

Fingerprint

Dive into the research topics of 'A function space analysis of finite neural networks with insights from sampling theory'. Together they form a unique fingerprint.

Cite this