15-463 Project #3: Face Morphing

Done by: Junjie Liang (junjieli)

13 October 2010

Overview

In this project, we were required to generate a morph sequence (two seconds of animation) where our faces were morphed into another student's face. We also computed the mean face of the class, and warped our face into the mean face's mesh. We wrote two functions:
morphed_im = morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac);
that produces a warp between im1 and im2 using point correspondences defined in im1_pts and im2_pts and the triangulation structure tri. The parameters warp_frac and dissolve_frac control shape warping and cross-dissolve respectively. We also had a function that computed an affine transformation A between two triangles:
A = computeAffine(tri1_pts, tri2_pts);

Algorithm

To help the computer match key features (eye to eye, nose to nose, etc.), we have to first define pairs of corresponding points by hand. Matlab has a useful feature, cpselect, which greatly helped with this process. I manually defined 76 pairs of correspondences between my face and Matt's face:

.

After that, the sets of points were averaged together to get their mean, from which a Delaunay triangulation is computed. We used the triangulation of the mean so that we get a triangulation that is nice for both our faces. 8 extra points were added to the border of the mesh, so that the entire picture is enclosed by the triangulation.

Then, it was just a simple matter of running morphed_im, setting warp_frac and dissolve_frac from 0 to 60 (in 1/60 increments) to generate the 61 frames of animation required.

Results - face morphing

My face:

Matt's face:

The morph:

Click here to see all 61 frames in their original size.

Results - class mean face

By averaging the point sets of the entire class, and morphing everyone's face into this "mean" structure, we can compute the "mean face" of the class, which is something like an average of everyones' faces. Here's the mean face for the class:

Surprisingly, the face does not look mean at all.

And here is my face, warped into the mean mesh:

And the mean face warped into my mesh: