Car Blending

Background

Out of all the projects we did this semester, face morphing was my favorite by far. When I finished the code, I started thinking about further applications of the algorithm. During the assignment, the best I came up with was a few silly animations (one of which is shown below), but I wanted to explore this idea further for my final project.

I eventually settled on cars as my target for this project. Although each car has similar features, some of them are impossible to blend together. Take the following two images for example:

There is a wealth of correspondence to choose from for the outer body shape (door handles, winshield lines, etc), but the wheels are impossible to blend, since they have a different number of spokes. Any attempt to simply blend the two results in a strange, ghosted image, as shown below (two different cars)

My challenge was to create novel, realistic-looking cars by combining components from an arbitrary number of cars.

Tools

I tried to use a reasonably large data set for the project, starting with 21 cars and eventually cutting it down to 15. I used around 150 correspondence points for each car, more than two thousand in total. I wanted to be able to expand the project in the future, so I created a small tool set to help with identifying initial correspondences and matching other images.

The points are split up into different files, each representing a different part of the car. A guide image shows the user the current set of points he must enter (teal), the current point he needs to enter (red), and the points entered thus far (yellow). I also included several scripts to automate the process of detecting which files are missing and prompt the user to input the data. Because of this, it will be very easy to add more cars in the future or expand the project to work with different objects.

Interface

I then built a GUI in MATLAB to allow the user to interact with the blender. The user can choose how much of each car they want to throw into the mix, select the body texture and wheels they want, and are presented with a novel composite car.

I used the points provided by the user to identify the wheels, extract them from the source image, and warp them to the new geometry. I created a triangulation from these points and generated a list of the pixels inside, copied those over to the morphed image, and blended the two.

Background Insertion

Another challenge I wanted to take on was to place the newly generated cars in a backdrop of my choosing, while keeping the shadows and window tint realistic.

The white background was easy to remove, but the shadow and window tint proved more difficult. I extracted both regions from the user input, then looked at them separately.

Here's an extracted window: I only wanted the background to show through the glass, not the pillars. I programatically identified the window color, calculated its effect on the white background, and applied this effect to my background on all window regions.

Extracted window

Glass identified

Novel car on a road with realistic shadow and window tint effects

VW bug warped to the average geometry