Face Morphing

William Keyes | 15-463 Project 3

A morph between my face and David's

Contents

Introduction

In omputational photography, morphing is the process of creating a smooth transition, or morph, from one picture to another. Unsurprisingly, in face morphing our two images our pictures of two faces.

By marking the positions of features, like the eyes or the nose, in each image, we can compute the transformation that moves each point to any other arbitrary point. Through some additional trickery, we "pull" the pixels of each image so that they follow the control points to the new positions.

Technical Details

Assuming that we have two images and a list of control points, morphing between the two images is relatively straighforward.

  1. Find the geometery of the desired result. For instance, if we want to evenly mix the two faces we find, for each pair of control points, the midpoint. If we're making one face look like the other, then we just use the control points for our target shape.
  2. Compute a triangulation of the desired geometery. Here, I used a Delaunay triangulation, which gives good results.
  3. Using the triangles that we just found, compute the affine transform that can convert a given triangle in one geometery to the coresponding triangle in the desired geometery. Do this for both source images.
  4. Look up the triangle in the desired geometery that contains each pixel of our result image. Using the transform for the triangle, find the coordinates of the pixel in the source image that maps to the current pixel of the result. Again, do this for both images.
  5. Copy the pixel in the source image into our result. If the point is not exactly on a pixel, create a pixel by interpolating between the pixels around the source coordinate.

Computing an Affine Transform

An affine transform in a single transformation that combines translation, rotation, scaling, and shearing. As a matrix, it is defined by six parameters. To compute it, we need three correspondances, or a total of six points. How convinient for us that six points can be obtained from two triangles!

I found the affine transform using a series of basis changes, which I've attempted to illustrate below. First, we find the matrix that converts a point in cartesian coordinates to coordinates defined by the sides of one triangle (Barycentric coordinates). Then we find the matrix that converts coordinates defined by the sides of the second triangle back to cartesian coordinates.

Now we apply a nice property of triangles - they are all basically the same. What this means is that if we have a point defined in terms of two sides of a triangle, the matching point in a different triangle has the same coordinates in terms of the same two sides of the new triangle. What we have are two different points that have the same relation with respect to their triangles. If we morph one triangle to the other, we also move one point to the other.

The end result of all of this is that if we combine the two coordinate (basis) changes, we have a single transformation that goes from pixel coordinates in one image to pixel coordinates in another - our affine transformation

Diagram illustrating the process described above.

Results

This is the average face of the class. The mean face, if you will. All faces were morphed to the average shape and then the pixel values were averaged together.

The average face of the class.

This is the mean face morphed to my facial structure.

The average face morphed to me geometery.

This is my face morphed to the average structure.

My face morphed to the average geometery.

Fun

Not feeling ambitious enough to try the more technical bells and whistle, I just had fun morphing things

For example, if we define two sets of control points on the same image, we can make squishy things! Blorp! Blorp!

A squishy dumbo octopus.

Next, I took a trip to the zoo. I saw a llama, a hideous gorilla, and a gazelle.

A trip through the zoo

Then, I mutated. Evolution in action! Unfortunately, the process left some artifacts...

From hand to foot and back again.