Learning Vector Quantized Shape Code for Amodal Blastomere Instance Segmentation

Won Dong Jang*, Donglai Wei, Xingxuan Zhang, Brian Leahy, Helen Yang, James Tompkin, Dalit Ben-Yosef, Daniel Needleman, Hanspeter Pfister

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Blastomere instance segmentation is important for analyzing embryos' abnormality. To measure the accurate shapes and sizes of blastomeres, their amodal segmentation is necessary. Amodal instance segmentation aims to recover an object's complete silhouette even when the object is not fully visible. For each detected object, previous methods directly regress the target mask from input features. However, images of an object under different amounts of occlusion should have the same amodal mask output, making it harder to train the regression model. To alleviate the problem, we propose to classify input features into intermediate shape codes and recover complete object shapes. First, we pre-train the Vector Quantized Variational Autoencoder (VQ-VAE) model to learn these discrete shape codes from ground truth amodal masks. Then, we incorporate the VQ-VAE model into the amodal instance segmentation pipeline with an additional refinement module. We also detect an occlusion map to integrate occlusion information with a backbone feature. As such, our network faithfully detects bounding boxes of amodal objects. On an internal embryo cell image benchmark, the proposed method outperforms previous state-of-the-art methods. To show generalizability, we show segmentation results on the public KINS natural image benchmark. Our method would enable accurate measurement of blastomeres in In Vitro Fertilization (IVF) clinics, potentially increasing the IVF success rate.

Original languageEnglish
Title of host publication2023 IEEE International Symposium on Biomedical Imaging, ISBI 2023
PublisherIEEE Computer Society
ISBN (Electronic)9781665473583
StatePublished - 2023
Event20th IEEE International Symposium on Biomedical Imaging, ISBI 2023 - Cartagena, Colombia
Duration: 18 Apr 202321 Apr 2023

Publication series

NameProceedings - International Symposium on Biomedical Imaging
ISSN (Print)1945-7928
ISSN (Electronic)1945-8452


Conference20th IEEE International Symposium on Biomedical Imaging, ISBI 2023


Dive into the research topics of 'Learning Vector Quantized Shape Code for Amodal Blastomere Instance Segmentation'. Together they form a unique fingerprint.

Cite this