15-463 Project 3: Face Morphing

Sometimes we find ourselves wanting to make art. Sometimes we settle with just making something that looks cool. For this project, we look at homogeneous images and how to create smooth transitions between them. With a little bit of human input, we can define the points of homogeniety in any two images and output an eerily realistic combination of the two images. In this project, we consider at frontal view portraits and the eery linear cross dissolve.

;

The Morph

One purpose of this project was to create an animation which does a morph through every student's face in the class. To do so, each student individually selected a set of points on their face, the locations of which we all use to correspond one face to another. My task was then to create 61 frames of animation from my face to another student's (Timothy's) face. I then combined the 61 frames to create this pretty animation.

;

One Mean Face

With all of this facial data, why not figure out what the average face is? Well, we can at least find the average according to a Carnegie Mellon computer science class. To find this great average face, I first averaged all control points to find the average face's mesh. Once this mesh was computed, I morphed each of the students' faces to it and averaged the resulting images. The mean image turned out to be rather short and round with a large, bulbous nose. I feel like I may have seen this kid on campus before...

I also morphed my face to the mean mesh. A few words of warning: this guy is not pretty.

Now that we know what my face looks like when it conforms to the average shape, why not take a look at how the average face looks when it conforms to my shape?

;

Caricatures (kind of...)

In addition to morphing my face to the mean, I attempted to extrapolate and find what my face looks like as it moves further from the mean. This is kind of like a caricature in the sense that it will attempt to accent what makes me different from the mean in a linear fashion.

The first image shifts the shape of my face 50% away from the mean.

I was also curious how shifting the texture of my face in the same manner would look. The only visible difference in the resulting image and the one above is the brightness. I believe this was due to the fact that my picture was taken next to the window and had better lighting than most others.

Now lets shift the shape of my face 100% away from the mean. This is so exciting.

Scary... I breifly thought of shifting further, but I was already in the realm of Herman Munster and too afraid to continue.

;

PCA, but not really

I really wanted to do principal component analysis on the images from this class, but I ran into some issues. At first, I wrote a short script to vectorize each image and concatenate them into a matrix. I then used princomp to get the coefficient matrix and saved the resultant basis images to disk. You can find the images here.

Being in color, the images were difficult to analyze and make conjectures about what the latent factors were in the basis for the set of input faces. I then decided to grayscale each of the images and repeat the same process to come up with a new basis that I may be able to reason about. Here are the resulting basis images in order of variance:

These results are not looking very good. Although all of the images are frontal view photographs of faces, each of the faces are slightly askew and some are completely off. Thus when the PCA coefficients were computed, some of the bases with highest component variance are simply determining whether they are not someone's face. The second image is a prime example, showing that a basis was formed to determine whether or not an image is Timothy's face since his face differs wildly in position and size from the others. The rest of the bases appear to give color and shape to the face, hair and shirt of the person in the photograph.

If I had more time, I would perform an additional normalization step before computing the PCA coefficients (and the mean face, for that matter). Ideally, I would take the control points at each person's eyes and transform the images and meshes so that they all match. This would not only improve the mean face, but it would also remove characteristics with high variance from the images and significantly improve the PCA.

Because of the poor basis, I did not bother to continue with matching new faces to this basis and I did not attempt to find PCA coefficients for the control points. These are two areas I wish to explore in the future.

;
{ }