› Project Description

  The purpose of this exercise is to explore image warping and blending. A non-rigid warping is achieved by dividing an object in triangles, where an affine transformation provides all possible degrees of freedom for the transformation. To calculate the transformation between images, a set of 43 point correspondences is provided and we calculate a Delaunay triangulation that is common to both images. Besides the 43 points provided for the face, we add four additional control points in each corner of the image, to provide a simple morphing/warping for the background. We add these points after calculating the triangulation for the face so as to avoid sub-optimal triangulations for the latter that use the added points. After adding the triangulation for the face, we add these control points and provide manual triangles for the background, following a simple criteria: each control point on the border of the face convex hull is connected to each of its neighbors and the nearest corner control point, forming two triangles. We could have done a more sophisticated approach by adding N control points along the border of the image and calculate a triangulation between the latter and points on the border of the face convex hull, removing afterwards triangles that only contain points on the latter. However, we found this simple method to give reasonable results. The final triangulation on all class faces morphed onto mean shape is here:

The mean face is obtained by averaging the appearance of all the faces morphed into the mean shape:

Yielding this lovely (not mean) fella:

One should note that the average looks male since only 2 females are present in class. The power of averaging shows itself in that the resulting face is very symmetric and thus aesthetically appeasing.

Back to top

› Morph to Scott

Nothing special here to note except the fact that the algorithm is especially picky about the triangulation used. Using mine/mean shape's provided bad results as it would place a triangle that caught a significant amount of background below the left ear and some flesh, so the result of the morph would put some of this background in the left cheek, giving it a Flesh eating diseasey look. As such, I resorted to Scott's in this and remainder morphs.

Back to top

› Me morphed into mean shape

  In this example, i morphed my appearance onto the mean shape:

It seems that i was smiling more than most my eyes were were frowned because of that so that also appears here.

Back to top

› Mean Shape morphed onto my shape

In this example, i morphed the mean face onto my shape:

Back to top

› Bells And Whistles

Caricatures: In this example, what i did was calculate the mean face for only the people that were not smiling {david; felix; garrick; greg; josip; minjalee; scott; youjia} and then morphed my appearance into towards that shape and extrapolated a little bit more (going further away from the convex combination that requires the shape to lie between this mean and my face). The extrapolated images (outside the convex combination scope) are marked by red squares on the corner. As can be seen, the extrapolation does not work all that well, reflecting the fact faces are only approximately a subspace and that we are sampling only very few of them.

PCA Basis: For the PCA basis, i coded the algorithm based just on the singular value decomposition, so as to make it faster. I applied this code to the images converted to BW (due to memory requirements). The mean and first 8 basis obtained are these (the variability obtained in the sample exceeds the one obtained by expectable factors such as illumination, glasses, teeth, skin color, facial hair; on our sample we get some variation from glasses and earrings, but the majority of it is just modeling several hair styles):

The evolution of the singular values (measure of the energy) is showed here. There is not a very rapid decay, further corroborates the hypothesis that 21 faces might not be a statistical sufficient sample of the face manifold, as this shows the matrix is not rank deficient.

After doing this, i projected my face (morphed onto average shape) into the basis and reconstructed it:

Here it is after adding come components of the second base vector. Voila, I Have earrings!

Median Face aka. "Combing" the mean face: In looking at the mean face, it seems like the hair outliers put some shading on the background. We can improve this image by computing a statistic that is more robust to outliers than the mean. There are many choices for this and a whole research field working on this and sparse representations, but here for simplicity, we provide results using the median. The image below shows a morph between the mean and median face and it suggests we could take a hybrid approach, handling the background and the face differently:

Back to top