TEXTure: Text-Guided Texturing of 3D Shapes

Elad Richardson*, Gal Metzer*, Yuval Alaluf, Raja Giryes, Daniel Cohen-Or

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

In this paper, we present TEXTure, a novel method for text-guided generation, editing, and transfer of textures for 3D shapes. Leveraging a pretrained depth-to-image diffusion model, TEXTure applies an iterative scheme that paints a 3D model from different viewpoints. Yet, while depth-to-image models can create plausible textures from a single viewpoint, the stochastic nature of the generation process can cause many inconsistencies when texturing an entire 3D object. To tackle these problems, we dynamically define a trimap partitioning of the rendered image into three progression states, and present a novel elaborated diffusion sampling process that uses this trimap representation to generate seamless textures from different views. We then show that one can transfer the generated texture maps to new 3D geometries without requiring explicit surface-to-surface mapping, as well as extract semantic textures from a set of images without requiring any explicit reconstruction. Finally, we show that TEXTure can be used to not only generate new textures but also edit and refine existing textures using either a text prompt or user-provided scribbles. We demonstrate that our TEXTuring method excels at generating, transferring, and editing textures through extensive evaluation, and further close the gap between 2D image generation and 3D texturing. Code is available via our project page: https://texturepaper.github.io/TEXTurePaper/.

Original languageEnglish
Title of host publicationProceedings - SIGGRAPH 2023 Conference Papers
EditorsStephen N. Spencer
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9798400701597
DOIs
StatePublished - 23 Jul 2023
Event2023 Special Interest Group on Computer Graphics and Interactive Techniques Conference, SIGGRAPH 2023 - Los Angeles, United States
Duration: 6 Aug 202310 Aug 2023

Publication series

NameProceedings - SIGGRAPH 2023 Conference Papers

Conference

Conference2023 Special Interest Group on Computer Graphics and Interactive Techniques Conference, SIGGRAPH 2023
Country/TerritoryUnited States
CityLos Angeles
Period6/08/2310/08/23

Funding

FundersFunder number
European Research CouncilERC-StG 757497
Israel Science Foundation2366/16, 2492/20

    Keywords

    • 3D Texturing
    • Diffusion Models

    Fingerprint

    Dive into the research topics of 'TEXTure: Text-Guided Texturing of 3D Shapes'. Together they form a unique fingerprint.

    Cite this