Project 3 - Face Morphing

Part 1: Defining Correspondences

I used the provided tool and created a set of correspondences between my own image and Steven Spielberg (after resizing and cropping):

My Picture

Steven Spielberg

After this, I used the Delaunay function from scipy to generate the triangulation. Here are the three triangulations (my picture, average, and steven Spielberg):

Triangulations (Mine, Average, Steven)

Part 2: Computing the Midway Face

After this, I implemented the affine warp function to create a midway face. To do this, I use the least squares method to solve for the affine transformation, which gives us a, b, c, d, e, f, g These then are used to create the A matrix, which can be then used on arbitrary triangles in the future. Finally, the last step for triangle warp is to use the linear interpolator to also warp the pixels within the triangles. Looping over the triangles, we can now warp every triangle to some triangulation. Here is the midway face:

Midway Face

Part 3: The Morph Sequence

After this, the natural next step is to create a morphing transition sequence between my original image and Steven Spielberg. To do this, we simply choose some alpha values and make weighted sums of both the shape (triangulation) and color (pixels) of the two images. Here are the results:

Morph Sequence (Tejas to Steven)

Part 4: Mean Face of Population

Once we have this, we can do some more statistical analysis using a full population's facial images. We choose the FEI Face dataset and take 400 images with 46 keypoints each (we also add corner keypoints). Then, we use the Delaunay function to generate the triangulation. We use the mean function to compute the mean of the triangulation. The mean is then used to warp the keypoints of the triangulation to the mean of the population images. Finally, we can use the interpolate function to warp the pixels within the triangles to the mean of the population. Here are the results:

Mean of Population

Examples of Population Images Warped to Mean

Next, we warp my own face to the population mean geometry and warp the population mean to my own geometry. This was done by creating keypoints matching the dataset's for my own picture and then using the warp_triangles function from before. These results look a little wonky just because it was hard to scale them so they look similar. Here are the results:

My Picture Warped to Mean

Mean Warped to My Geometry

Part 5: Caricatures

Finally, we can extrapolate my face from the population mean by choosing alpha values greater than 1 or less than 0. This creates an interesting caricature effect (some median filter is used to remove artifacts):

Alpha: -0.5

Alpha: 2.0

Bells and Whistles: Morphing Gender

I chose to do the first bell/whistle, which was trying to convert my face's gender into another gender using a specific population mean. For this, I used this image (displayed below). Then, I used the earlier techniques to first blend the shapes, then the colors, then both at once. Here is the result:

Shape Only

Appearance Only

Both

And so I conclude this project!