Concept Decomposition for Visual Exploration and Inspiration

Yael Vinker, Andrey Voynov, Daniel Cohen-Or, Ariel Shamir

Research output: Contribution to journalArticlepeer-review

Abstract

A creative idea is often born from transforming, combining, and modifying ideas from existing visual examples capturing various concepts. However, one cannot simply copy the concept as a whole, and inspiration is achieved by examining certain aspects of the concept. Hence, it is often necessary to separate a concept into different aspects to provide new perspectives. In this paper, we propose a method to decompose a visual concept, represented as a set of images, into different visual aspects encoded in a hierarchical tree structure. We utilize large vision-language models and their rich latent space for concept decomposition and generation. Each node in the tree represents a sub-concept using a learned vector embedding injected into the latent space of a pretrained text-to-image model. We use a set of regularizations to guide the optimization of the embedding vectors encoded in the nodes to follow the hierarchical structure of the tree. Our method allows to explore and discover new concepts derived from the original one. The tree provides the possibility of endless visual sampling at each node, allowing the user to explore the hidden sub-concepts of the object of interest. The learned aspects in each node can be combined within and across trees to create new visual ideas, and can be used in natural language sentences to apply such aspects to new designs. Project page: https://inspirationtree.github.io/inspirationtree/

Original languageEnglish
Article number241
JournalACM Transactions on Graphics
Volume42
Issue number6
DOIs
StatePublished - 4 Dec 2023

Keywords

  • concept decomposition
  • personalization
  • visual inspiration and exploration

Fingerprint

Dive into the research topics of 'Concept Decomposition for Visual Exploration and Inspiration'. Together they form a unique fingerprint.

Cite this