3D Alignment of Face in a Single Image




Given a single image of a frontal or profile face, our system automatically recovers the 3D structure and 3D pose of the face. Our model consists of a sparse set of 3D landmarks and the view-based patches associated with each landmark. We introduce a novel EM-based appraoch that deforms the 3D model to fit noisy 2D feature points located on the image. In the fitting process, the self-occluded part and the depth of the face are treated as hidden variables, the image observation noise is regularized by 3D face shape prior, and 3D pose and 3D shape are estimated iteratively. Our technique is appliable to generic faces with a large range of view changes.

Related Publications:

Gu, L. and Kanade, T.
3D Alignment of Face in a Single Image
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, 2006.
(oral presentation)