UNIQ: Uniform Noise Injection for Non-Uniform Qantization of Neural Networks

Chaim Baskin, Natan Liss, Eli Schwartz, Evgenii Zheltonozhskii, Raja Giryes, Alex M. Bronstein, Avi Mendelson

Research output: Contribution to journalArticlepeer-review

Abstract

We present a novel method for neural network quantization. Our method, named UNIQ, emulates a non-uniform k-quantile quantizer and adapts the model to perform well with quantized weights by injecting noise to the weights at training time. As a by-product of injecting noise to weights, we find that activations can also be quantized to as low as 8-bit with only a minor accuracy degradation. Our non-uniform quantization approach provides a novel alternative to the existing uniform quantization techniques for neural networks. We further propose a novel complexity metric of number of bit operations performed (BOPs), and we show that this metric has a linear relation with logic utilization and power. We suggest evaluating the trade-off of accuracy vs. complexity (BOPs). The proposed method, when evaluated on ResNet18/34/50 and MobileNet on ImageNet, outperforms the prior state of the art both in the low-complexity regime and the high accuracy regime. We demonstrate the practical applicability of this approach, by implementing our non-uniformly quantized CNN on FPGA.

Original languageEnglish
Article number3444943
JournalACM Transactions on Computer Systems
Volume37
Issue number1-4
DOIs
StatePublished - Jun 2021

Keywords

  • Deep learning
  • efficient deep learning
  • neural networks
  • quantization

Fingerprint

Dive into the research topics of 'UNIQ: Uniform Noise Injection for Non-Uniform Qantization of Neural Networks'. Together they form a unique fingerprint.

Cite this