On divergence approximations for unsupervised training of deep denoisers based on Stein's unbiased risk estimator

Shakarim Soltanayev, Raja Giryes, Se Young Chun, Yonina C. Eldar

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Recently, there have been several works on unsupervised learning for training deep learning based denoisers without clean images. Approaches based on Stein's unbiased risk estimator (SURE) have shown promising results for training Gaussian deep denoisers. However, their performance is sensitive to hyper-parameter selection in approximating the divergence term in the SURE expression. In this work, we briefly study the computational efficiency of Monte-Carlo (MC) divergence approximation over recently available exact divergence computation using backpropagation. Then, we investigate the relationship between smoothness of nonlinear activation functions in deep denoisers and robust divergence term approximations. Lastly, we propose a new divergence term that does not contain hyper-parameters. Both unsupervised training methods yield comparable performance to supervised training methods with ground truth for denoising on various datasets. While the former method still requires roughly tuned hyper parameter selection, the latter method removes the necessity of choosing one.

Original languageEnglish
Title of host publication2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3592-3596
Number of pages5
ISBN (Electronic)9781509066315
DOIs
StatePublished - May 2020
Event2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Barcelona, Spain
Duration: 4 May 20208 May 2020

Publication series

NameICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume2020-May
ISSN (Print)1520-6149

Conference

Conference2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020
Country/TerritorySpain
CityBarcelona
Period4/05/208/05/20

Keywords

  • Deep learning
  • Denoising
  • Divergence term
  • SURE
  • Unsupervised training

Fingerprint

Dive into the research topics of 'On divergence approximations for unsupervised training of deep denoisers based on Stein's unbiased risk estimator'. Together they form a unique fingerprint.

Cite this