HIERARCHICAL TIMBRE-PAINTING AND ARTICULATION GENERATION

Michael Michelashvili, Lior Wolf

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

We present a fast and high-fidelity method for music generation, based on specified f0 and loudness, such that the synthesized audio mimics the timbre and articulation of a target instrument. The generation process consists of learned source-filtering networks, which reconstruct the signal at increasing resolutions. The model optimizes a multi-resolution spectral loss as the reconstruction loss, an adversarial loss to make the audio sound more realistic, and a perceptual f0 loss to align the output to the desired input pitch contour. The proposed architecture enables high-quality fitting of an instrument, given a sample that can be as short as a few minutes, and the method demonstrates state-of-the-art timbre transfer capabilities. Code and audio samples are shared at https://github.com/mosheman5/timbre_painting.

Original languageEnglish
Title of host publicationProceedings of the 21st International Society for Music Information Retrieval Conference, ISMIR 2020
EditorsJulie Cumming, Jin Ha Lee, Brian McFee, Markus Schedl, Johanna Devaney, Johanna Devaney, Cory McKay, Eva Zangerle, Timothy de Reuse
PublisherInternational Society for Music Information Retrieval
Pages216-222
Number of pages7
ISBN (Electronic)9780981353708
StatePublished - 2020
Event21st International Society for Music Information Retrieval Conference, ISMIR 2020 - Virtual, Online, Canada
Duration: 11 Oct 202016 Oct 2020

Publication series

NameProceedings of the 21st International Society for Music Information Retrieval Conference, ISMIR 2020

Conference

Conference21st International Society for Music Information Retrieval Conference, ISMIR 2020
Country/TerritoryCanada
CityVirtual, Online
Period11/10/2016/10/20

Funding

FundersFunder number
Guy Harries and Adam Polyak
European Research Council
Horizon 2020ERC CoG 725974

    Fingerprint

    Dive into the research topics of 'HIERARCHICAL TIMBRE-PAINTING AND ARTICULATION GENERATION'. Together they form a unique fingerprint.

    Cite this