Meta Internal Learning

Internal learning for single-image generation is a framework where a generator is trained to produce novel images based on a single image. Since these models are trained on a single image, they are limited in their scale and application. To overcome these issues, we propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively. In the presented meta-learning approach, a single-image GAN model is generated given an input image, via a convolutional feedforward hypernetwork f. This network is trained over a dataset of images, allowing for feature sharing among different models and for interpolation in the space of generative models. The generated single-image model contains a hierarchy of multiple generators and discriminators. Therefore, the meta-learner needs to be trained in an adversarial manner, which requires careful design choices that we justify by a theoretical analysis. Our results show that the models obtained are as suitable as single-image GANs for many common image applications, and significantly reduce training time per image, without loss in performance, and introduce novel capabilities, such as interpolation and feedforward modeling of novel images.