Nonrigid 3D from video with a super-fast SVD

Matthew Brand
MERL

Time and Place

1305 Newell-Simon Hall
Refreshments 3:15 pm
Talk 3:30 pm

Abstract

3D morphable models play a large role in vision research and commercial animation/special effects, but they are quite difficult to acquire or even manually construct. We'll look at the problem of estimating the 3D shape, motion, and articulations of a nonrigid surface such a face directly from intensity variations in video. This is a multilinear factorization problem with inhomogeneous anisotropic noise induced by the surface texture. I'll identify the key subspaces for this problem and show how they can be estimated by integrating out all uncertainty due to noise. This yields a combined 2D tracking+3D factorization algorithm whose performance with a single uncalibrated low-res camera is competitive with that of high-end motion-capture rigs.

Although very effective, subspace methods do not scale well because they are built on the computationally expensive thin singular value decomposition. As problem sizes grow, the quadratically growing compute time of the SVD will become prohibitive, especially if the data matrix does not fit in memory. I'll show how to compute a rank-revealing thin SVD from streaming data in strictly linear time with sublinear storage costs. This subspace-updating method offers enormous speed-ups and a principled way to handle missing data such as occlusions.

Speaker Biography

Matthew Brand studied neuroscience, cogitive science, & computer science at Yale and Northwestern Universities, taught at MIT's Media Lab, and is now a research scientist at MERL, where his work on the perception, modeling, and mimicry of human expressive behavior has won several industrial and academic awards.

Speaker Appointments

For appointments, please contact Yanxi Liu (yanxi@cs.cmu.edu).