Separable Spatiotemporal Priors for Convex Reconstruction of

Time-varying 3D Point Clouds

Tomas Simon 1             Jack Valmadre 2

Iain Matthews 1,3          Yaser Sheikh 1


1 The Robotics Institute

Carnegie Mellon University

3 Disney Research


2 Queensland University of Technology, Australia

CSIRO, Australia


Reconstructing 3D motion data is highly under-constrained due to several common sources of data loss during measurement, such as projection, occlusion, or miscorrespondence. We present a statistical model of 3D motion data, based on the Kronecker structure of the spatiotemporal covariance of natural motion, as a prior on 3D motion. This prior is expressed as a matrix normal distribution, composed of separable and compact row and column covariances. We relate the marginals of the distribution to the shape, trajectory, and shape-trajectory models of prior art. When the marginal shape distribution is not available from training data, we show how placing a hierarchical prior over shapes results in a convex MAP solution in terms of the trace-norm. The matrix normal distribution, fit to a single sequence, outperforms state-of-the-art methods at reconstructing 3D motion data in the presence of significant data loss, while providing covariance estimates of the imputed points.


Separable Spatiotemporal Priors for Convex Reconstruction of Time-varying 3D Point Clouds

Tomas Simon, Jack Valmadre, Iain Matthews, and Yaser Sheikh

European Conference on Computer Vision, September 2014

[ Paper (PDF, 14 MB) ] [ BibTeX ]

Copyright notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Source Code

Coming soon.