An adversarial learning approach to medical image synthesis for lesion detection

Liyan Sun, Jiexiang Wang, Yue Huang, Xinghao Ding*, Hayit Greenspan, John Paisley

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

74 Scopus citations


The identification of lesion within medical image data is necessary for diagnosis, treatment and prognosis. Segmentation and classification approaches are mainly based on supervised learning with well-paired image-level or voxel-level labels. However, labeling the lesion in medical images is laborious requiring highly specialized knowledge. We propose a medical image synthesis model named abnormal-to-normal translation generative adversarial network (ANT-GAN) to generate a normal-looking medical image based on its abnormal-looking counterpart without the need for paired training data. Unlike typical GANs, whose aim is to generate realistic samples with variations, our more restrictive model aims at producing a normal-looking image corresponding to one containing lesions, and thus requires a special design. Being able to provide a 'normal' counterpart to a medical image can provide useful side information for medical imaging tasks like lesion segmentation or classification validated by our experiments. In the other aspect, the ANT-GAN model is also capable of producing highly realistic lesion-containing image corresponding to the healthy one, which shows the potential in data augmentation verified in our experiments.

Original languageEnglish
Article number8950113
Pages (from-to)2303-2314
Number of pages12
JournalIEEE Journal of Biomedical and Health Informatics
Issue number8
StatePublished - Aug 2020


FundersFunder number
CCF-Tencent open fund
Columbia University
National Natural Science Foundation of China61671309, 61571382, 81671766, U1605252, 81671674, 61571005
Natural Science Foundation of Fujian Province2017J01126
China Scholarship Council201806310090
Fundamental Research Funds for the Central Universities20720180059, 20720190116, 20720160075


    • Medical image synthesis
    • generative adversarial network
    • unsupervised learning


    Dive into the research topics of 'An adversarial learning approach to medical image synthesis for lesion detection'. Together they form a unique fingerprint.

    Cite this