DeepISP: Toward learning an end-to-end image processing pipeline

Eli Schwartz*, Raja Giryes, Alex M. Bronstein

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

147 Scopus citations

Abstract

We present DeepISP, a full end-to-end deep neural model of the camera image signal processing pipeline. Our model learns a mapping from the raw low-light mosaiced image to the final visually compelling image and encompasses low-level tasks, such as demosaicing and denoising, as well as higher-level tasks, such as color correction and image adjustment. The training and evaluation of the pipeline were performed on a dedicated data set containing pairs of low-light and well-lit images captured by a Samsung S7 smartphone camera in both raw and processed JPEG formats. The proposed solution achieves the state-of-the-art performance in objective evaluation of peak signal-to-noise ratio on the subtask of joint denoising and demosaicing. For the full end-to-end pipeline, it achieves better visual quality compared to the manufacturer ISP, in both a subjective human assessment and when rated by a deep model trained for assessing image quality.

Original languageEnglish
Article number8478390
Pages (from-to)912-923
Number of pages12
JournalIEEE Transactions on Image Processing
Volume28
Issue number2
DOIs
StatePublished - Feb 2019

Funding

FundersFunder number
ERC-StG
Horizon 2020 Framework Programme757497

    Keywords

    • ISP
    • Image processing
    • color correction
    • deep learning
    • demosaicing
    • denoising

    Fingerprint

    Dive into the research topics of 'DeepISP: Toward learning an end-to-end image processing pipeline'. Together they form a unique fingerprint.

    Cite this