Weakly supervised 2D human pose transfer

Qian Zheng, Yajie Liu, Zhizhao Lin, Dani Lischinski, Daniel Cohen-Or, Hui Huang

Research output: Contribution to journalArticlepeer-review

Abstract

We present a novel method for pose transfer between two 2D human skeletons. When the bone lengths and proportions between the two skeletons are significantly different, pose transfer becomes a challenging task, which cannot be accomplished by simply copying the joint positions or the bone directions. Our data-driven approach utilizes a deep neural network trained, in a weakly supervised fashion, to encode a skeleton into two separate latent codes, one representing its pose, and another representing the skeleton’s proportions (skeleton-ID). The network is given two skeletons, and learns to combine the pose of one with the skeleton-ID of the other. Lacking supervision on the poses, we develop a novel loss that qualitatively compares poses of different skeletons. We evaluate the performance of our method on a large set of poses. The advantages of avoiding supervision are demonstrated by showing transfer of extreme poses, as well as between uncommon skeleton proportions.

Original languageEnglish
Article number210103
JournalScience China Information Sciences
Volume64
Issue number11
DOIs
StatePublished - Nov 2021
Externally publishedYes

Keywords

  • human skeleton
  • pose transfer
  • weak supervision

Fingerprint

Dive into the research topics of 'Weakly supervised 2D human pose transfer'. Together they form a unique fingerprint.

Cite this