Multi-view scene flow estimation: A view centered variational approach

Tali Basha*, Yael Moses, Nahum Kiryati

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

We present a novel method for recovering the 3D structure and scene flow from calibrated multi-view sequences. We propose a 3D point cloud parametrization of the 3D structure and scene flow that allows us to directly estimate the desired unknowns. A unified global energy functional is proposed to incorporate the information from the available sequences and simultaneously recover both depth and scene flow. The functional enforces multi-view geometric consistency and imposes brightness constancy and piecewise smoothness assumptions directly on the 3D unknowns. It inherently handles the challenges of discontinuities, occlusions, and large displacements. The main contribution of this work is the fusion of a 3D representation and an advanced variational framework that directly uses the available multi-view information. This formulation allows us to advantageously bind the 3D unknowns in time and space. Different from optical flow and disparity, the proposed method results in a nonlinear mapping between the images' coordinates, thus giving rise to additional challenges in the optimization process. Our experiments on real and synthetic data demonstrate that the proposed method successfully recovers the 3D structure and scene flow despite the complicated nonconvex optimization problem.

Original languageEnglish
Pages (from-to)6-21
Number of pages16
JournalInternational Journal of Computer Vision
Volume101
Issue number1
DOIs
StatePublished - Jan 2013

Keywords

  • 3D structure
  • Multiple view
  • Scene flow

Fingerprint

Dive into the research topics of 'Multi-view scene flow estimation: A view centered variational approach'. Together they form a unique fingerprint.

Cite this