Gradient Adjusting Networks for Domain Inversion

Erez Sheffi, Michael Rotman, Lior Wolf*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


StyleGAN2 was demonstrated to be a powerful image generation engine that supports semantic editing. However, in order to manipulate a real-world image, one first needs to be able to retrieve its corresponding latent representation in StyleGAN’s latent space that is decoded to an image as close as possible to the desired image. For many real-world images, a latent representation does not exist, which necessitates the tuning of the generator network. We present a per-image optimization method that tunes a StyleGAN2 generator such that it achieves a local edit to the generator’s weights, resulting in almost perfect inversion, while still allowing image editing, by keeping the rest of the mapping between an input latent representation tensor and an output image relatively intact. The method is based on a one-shot training of a set of shallow update networks (aka. Gradient Modification Modules) that modify the layers of the generator. After training the Gradient Modification Modules, a modified generator is obtained by a single application of these networks to the original parameters, and the previous editing capabilities of the generator are maintained. Our experiments show a sizable gap in performance over the current state of the art in this very active domain. Our code is available at

Original languageEnglish
Title of host publicationImage Analysis - 23rd Scandinavian Conference, SCIA 2023, Proceedings
EditorsRikke Gade, Michael Felsberg, Joni-Kristian Kämäräinen
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages20
ISBN (Print)9783031314377
StatePublished - 2023
Event23nd Scandinavian Conference on Image Analysis, SCIA 2023 - Lapland, Finland
Duration: 18 Apr 202321 Apr 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13886 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference23nd Scandinavian Conference on Image Analysis, SCIA 2023


Dive into the research topics of 'Gradient Adjusting Networks for Domain Inversion'. Together they form a unique fingerprint.

Cite this