A-Muze-Net: Music Generation by Composing the Harmony Based on the Generated Melody

Or Goren*, Eliya Nachmani, Lior Wolf

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


We present a method for the generation of Midi files of piano music. The method models the right and left hands using two networks, where the left hand is conditioned on the right hand. This way, the melody is generated before the harmony. The Midi is represented in a way that is invariant to the musical scale, and the melody is represented, for the purpose of conditioning the harmony, by the content of each bar, viewed as a chord. Finally, notes are added randomly, based on this chord representation, in order to enrich the generated audio. Our experiments show a significant improvement over the state of the art for training on such datasets, and demonstrate the contribution of each of the novel components.

Original languageEnglish
Title of host publicationMultiMedia Modeling - 28th International Conference, MMM 2022, Proceedings
EditorsBjörn Þór Jónsson, Cathal Gurrin, Minh-Triet Tran, Duc-Tien Dang-Nguyen, Anita Min-Chun Hu, Binh Huynh Thi Thanh, Benoit Huet
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages12
ISBN (Print)9783030983574
StatePublished - 2022
Event28th International Conference on MultiMedia Modeling, MMM 2022 - Phu Quoc, Viet Nam
Duration: 6 Jun 202210 Jun 2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13141 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference28th International Conference on MultiMedia Modeling, MMM 2022
Country/TerritoryViet Nam
CityPhu Quoc


FundersFunder number
Horizon 2020 Framework ProgrammeERC CoG 725974
European Research Council


    • Midi processing
    • Music generation
    • Recurrent neural networks


    Dive into the research topics of 'A-Muze-Net: Music Generation by Composing the Harmony Based on the Generated Melody'. Together they form a unique fingerprint.

    Cite this