Project 2 – Eulerian Video Magnification

Alex Limpaecher

 

Overview

For this project I was using Eulerian Video Magnification to visualize difficult to detect changes in a video. Primarily I used this to detect and visualizing pulses in videos.

 

Process

Laplacian Pyramids

The first step was to create a Laplacian Pyramid. Each level of the Laplacian pyramid has a different spatial frequency. I created the laplacian pyramid as follows:

 

First I created a gaussian pyramid where each additional layer was both blurred and one half the size of the previous layer.

 

Then I subtracted one level of the gaussian pyramid from the next lowest level of the gaussian pyramid to get the laplacian.

 

These gaussian pyramids were done on the massive scale for an array of image that made up the whole video.

Temporal Filtering

Once the Laplacian Pyramid was created I could temporarily filter each layer of the Laplacian Pyramid. By running the fast Fourier transform along the time dimension of the spatially filtered arrays, I could convert the whole pyramids into the frequency domain.

 

Filtering Out Frequencies

In the frequency domain I could then filter. I did this by zeroing out different frequencies. Deciding which frequencies to zero out was the most difficult part.

 

The matlab function fft returns a one dimensional vector of frequencies. I needed to figure out which frequencies to amplify and which ones to keep the same. For the face for example I choose to filter out the frequencies from .83 to 1.   I needed to figure out where in the fft return vector I needed to filter.

 

The formula a came up from converting to Hz to location in the vector was as follows.

 

vectorLocation = NumberOfFrequencies/Frames Per Second * Hz

 

Thus for the face the locations I wanted to keep were:

520 / 30 * .83 ~ 14

520 / 30 * 1 ~ 18

 

It was important to note that since fft returned a symmetric matrix and thus I needed to be careful to not zero out the symmetry.

 

Thus I would zero out all but:

1 – Because it that is where the direct current was stored

14 – 18 the frequencies that I wanted

502 – 506 because thatŐs where the frequencies that I wanted were mirrored.

 

Amplitude

Once I filtered the frequency domain that I wanted, I added it back to the original frequency domain. It became clear that important spatial frequencies were those on the top of the pyramid. This is where most of the movement had been blurred out, and only the major color changes remained.

 

Efficiency

At first my program had a very very long run time. This is because I was filtering every level of the laplacian pyramid. However this was unnecessary and inefficient. Since I was only paying attention to the higher layers of the laplacian pyramid and was multiplying the rest of the layers by 0, I could ignore the rest of the laplacian pyramid. This was made even more efficient because the higher layers were the smaller layers of the pyramid.

 

Results

For my results I converted the images into the YIQ space. For efficiency, I only ran my filtering on the Q space, which gives everything a purple glow.

 

Movie

Lower Frequency

Higher Frequency

Laplacian Layers

Layer Amplitudes

Face

.83Hz

1Hz

6

6th layer * 20

All others * 0

Baby 2

1.5Hz

3Hz

8

8th Layer * 10

7th Layer * 10

All others * 0

Hand

.83Hz

2Hz

8

8th Layer * 10

7th Layer * 10

All others * 0

 

 

Face

The Eulerian Video Magnification paper put the faces frequency between .83Hz and 1Hz. I found this to be the case.

 

 

Baby2

While the paper put the babies pulse at 2.33Hz to 2.67 Hz, I tried many different ranges. In the end I settled on 1.5Hz and 3Hz. Oddly enough, I get a pulse at the very beginning of the video, however it becomes only very suble a couple of moments in. I am not sure what causes this. My only guess is that I have my amplitudes not adjusted correctly. Also I wonder if the movement of the babies face is causing some problems with my code.

 

 

Hand

For my extra video I recorded my own hand. I measured my own pulse and found it to be 75 beats per minute. I decided on filtering between .83 Hz and 2Hz. WhatŐs very interesting is that you can see the heart beat pattern, not just the pulse. I found this pretty exciting.