TY - GEN
T1 - HIERARCHICAL TIMBRE-PAINTING AND ARTICULATION GENERATION
AU - Michelashvili, Michael
AU - Wolf, Lior
N1 - Publisher Copyright:
© Michael Michelashvili, Lior Wolf.
PY - 2020
Y1 - 2020
N2 - We present a fast and high-fidelity method for music generation, based on specified f0 and loudness, such that the synthesized audio mimics the timbre and articulation of a target instrument. The generation process consists of learned source-filtering networks, which reconstruct the signal at increasing resolutions. The model optimizes a multi-resolution spectral loss as the reconstruction loss, an adversarial loss to make the audio sound more realistic, and a perceptual f0 loss to align the output to the desired input pitch contour. The proposed architecture enables high-quality fitting of an instrument, given a sample that can be as short as a few minutes, and the method demonstrates state-of-the-art timbre transfer capabilities. Code and audio samples are shared at https://github.com/mosheman5/timbre_painting.
AB - We present a fast and high-fidelity method for music generation, based on specified f0 and loudness, such that the synthesized audio mimics the timbre and articulation of a target instrument. The generation process consists of learned source-filtering networks, which reconstruct the signal at increasing resolutions. The model optimizes a multi-resolution spectral loss as the reconstruction loss, an adversarial loss to make the audio sound more realistic, and a perceptual f0 loss to align the output to the desired input pitch contour. The proposed architecture enables high-quality fitting of an instrument, given a sample that can be as short as a few minutes, and the method demonstrates state-of-the-art timbre transfer capabilities. Code and audio samples are shared at https://github.com/mosheman5/timbre_painting.
UR - http://www.scopus.com/inward/record.url?scp=85108594517&partnerID=8YFLogxK
M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???
AN - SCOPUS:85108594517
T3 - Proceedings of the 21st International Society for Music Information Retrieval Conference, ISMIR 2020
SP - 216
EP - 222
BT - Proceedings of the 21st International Society for Music Information Retrieval Conference, ISMIR 2020
A2 - Cumming, Julie
A2 - Lee, Jin Ha
A2 - McFee, Brian
A2 - Schedl, Markus
A2 - Devaney, Johanna
A2 - Devaney, Johanna
A2 - McKay, Cory
A2 - Zangerle, Eva
A2 - de Reuse, Timothy
PB - International Society for Music Information Retrieval
T2 - 21st International Society for Music Information Retrieval Conference, ISMIR 2020
Y2 - 11 October 2020 through 16 October 2020
ER -