15-463: Computational Photography

Project 2: Seam Carving

Overview

We attempted to morph faces using feature matching, transformations, and blending.

To do the morphing, we first defined a set of 43 corresponding features across all faces. Then a Delaunay triangulation was found using the average of the points from the initial face and the final face. This triangulation would define which triangular group of pixels from the initial face would morph into the set of pixels in the final face.

To perform incremental morphing, we defined an intermediate shape that both faces would be morphed to. This intermediate shape was a weighted average of the features from both faces. Then for each triangle, I found a transformation matrix from the both faces to the intermediate face.

Then for each point in the new image, I found which triangle it belonged to. I used the inverse of the transformation matrix to determine which point in the initial face and the final face generated this point. Using interpolation, I determined a color for this pixel from the original images. If a point was outside a triangle, then it kept the original value. As we will see later, this will have some bad effects on the resulting morph.

Then the colors were cross-dissolved, again as a weighted average in order to yield an incremental change for a movie.

I repeated this algoirthm for 60 equal intervals to get 61 frames, which were then assembled into a movie using mencoder and then uploaded to YouTube.

For the mean face portion of the assignment, I calculated a average shape and then morphed all the faces to it. Then I took the average of the colors from each face. One face was omitted because it only had 42 feature points not the required 43.

Original data can be found here.

Results

Originals

Initial image Final image
initial face final face

Morph video

Towards the middle of the video, we can see that the hair from the first face is still visible but faded. This is because the initial face was larger than the final face, so it's features had to shrink towards the center. This raises the question of what the algorithm should do to fill in the hair. In the current implementation, because this area lies outside the triangle, the image is simply copied from the initial face, and then cross dissolved with what happens to be the background in the final face. This causes the fading effect. Another method that may be more effective would be to define additional points to allow the background to morph as well and hopefully fill up the void left by the missing hair.

Mean face

My implementation used three scripts to produce this result. First, I morphed all the faces into the mean shape (meandriver1.m). These results are located here as all the morph_*.jpg files. Then all the faces were averaged (meandriver2.m). Then the various morphs to/from the mean were performed (meandriver3.m).

mean face

Mean features

My face morphed to mean shape

my face morphed to mean face

Mean face morphed to my shape

mean face morphed to my face