Efficient proximal mapping of the 1-path-norm of shallow networks

We demonstrate two new important properties of the 1-path-norm of shallow neural networks. First, despite its non-smoothness and non-convexity it allows a closed form proximal operator which can be efficiently computed, allowing the use of stochastic proximal-gradient-type methods for regularized empirical risk minimization. Second, when the activation functions is differentiable, it provides an upper bound on the Lipschitz con_stant of the network. Such bound is tighter than the trivial layer-wise product of Lipschitz con_stants, motivating its use for training networks robust to adversarial perturbations. In practical experiments we illustrate the advantages of us_ing the proximal mapping and we compare the robustness-accuracy trade-off induced by the 1- path-norm, L1-norm and layer-wise constraints on the Lipschitz constant (Parseval networks).