In this assignment I produce a "morph" animation from my face to the faces of other students in class. Additionally I explore computing the "mean face" of the class using aligned images, preforming masculitizing and feminizing transforms and computing a PCA basis for the face space for additional trasformations.
A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin tochin, ears to ears, etc., to get the smoothest transformations possible.
The correspondence should map eyes to eyes, mouth to mouth, chin tochin, ears to ears, etc., to get the smoothest transformations possible. Corresponding points are hand labeled according to the scheme shown below. This is labeling is consistent within our class and across multiple years.
Now we need a triangular mesh that will beused for morphing. A Delaunay triangulation (see MATLAB function dalaunay and related functions) is used on the mean (intra-class) shape to define the triangular mesh shown below.
I implemented the function:
morphed_im = morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac);
that produces a warp between im1 and im2 using point correspondences defined in im1_pts and im2_pts (whichare both n-by-2 matrices of (x,y) locations) and the triangulation structure tri. The parameters warp_frac and dissolve_frac control shape warping and cross-dissolve, respectively. In particular, images im1 and im2 are first warped into an intermediate shape configuration controlled by warp_frac, and then cross-dissolved according to dissolve_frac. For interpolation, both parameters lie in the range [0,1].
Given a new intermediate shape, the main task is implementing an affine warp for each triangle in the triangulation from the original images into this new shape.This will involve computing an affine transformation matrix A betweentwo triangles:
A = computeAffine(tri1_pts,tri2_pts)
This is performed using linear algebra as follows.
Let A be a data matrix composed of the x and y
coordinates of tri1_pts.
Let b be a vector composed of the x and y coordinates of
tri2_pts.
Let t be the reordered affine transform you are solving for.
Solve: t = inv(A)*b
 |
 |
We can compute the mean face of 15-463 students.This involves: 1) computing the average shape, 2) warping all faces into that shape, and 3) averaging the colors together. This can be performed for different subsets of the class. The mean faces displayed below are for several semesters worth of face data. I also normalized the "Mean Face" based on a 50-50 male/female global population split.
 |
 |
 |
| Mean Face | Mean Male Face | Mean Female Face |
 |
 |
 |
 |
| Average Face -- Warped to my Geometry | My Face -- Warped to Average Geometry | My Face -- Warped to Average Male Geometry | My Face -- Warped to Average Female Geometry |
Using the average male and female faces, we are able to compute deviations between male and female in both shape and color (color differences shown below).
We are then able to deform shape and/or color of an input face in the direction of either "more female" or "more male". This is shown on my face as shown in the image below, where the moving up or left corresponds to "more female" colors/shape, and moving right or down corresponds to "more male" color/shape.
We use Principal Componenet Anaylsis to determine a new orthogonal basis in which to represent our faces. Below are grayscale versions of the first five "eigenfaces" of the mean shape.
I projected my face into both the color and shape subspaces and manipulated the projection coefficients before reconstructing. The animations below iterate through the first five principal compoenents (in color on the left, and space on the right). Precisely what each component controls is difficult to ascertain. This creates some interesting transforms, but it makes it difficult to precisely control for a specific trait.
