Ruslan Aiginin - Project 2

Eulerian Video Magnification

For this assignment, we had to recreate the work from the MIT paper from 2012 "Eulerian Video Magnification for Revealing Subtle Changes in the World", the idea behind which is to allow people to see what is normally invisible by amplifying certain frequencies in the video. So while we can't see things like pulse or blood flow in a typical video with a human, it's actually there, but the frequency is too weak for us to see.

Results

Frequencies: [0.83, 1.0]

Frequencies: [0.5, 0.7] (video shortened for faster computation)

 

This is the video of my classmate Oleg's hand. The video, though of poor quality, shows his pulse. Frequencies [0.5, 0.7].

Approach

I used the following approach to complete the assignment.

1) Created a Gaussian pyramid of the timeline of images by resizing the images by half subsequently.

2) Created a Laplacian pyramid by subtracting levels of Gaussian pyramid with upscaling by 2.

3) For each level of Laplacian pyramid, I filtered all the time vectors (which show how one a single pixel changes over time). I filtered vectors by applying the Fast Fourier transform on them, then multiplying by the filtering frequency vector. I amplified the frequencies I needed by 10 and then converted it back from the frequency domain, using Inverse fast Fourier transform. This was the result I needed.

4) Then, to rebuild the video, I recreated the image using pyramids, resizing and adding the filtered layers. The resulting images were written into a video file.

Bells and Whistles

None this time.

Difficulties

This project took an insane amount of time and effort for me. I faced a lot of diffuculties, and the more time I spent without getting any result, the more my frustration grew. However, I was really pleased when I finally completed it. I think I also received of good amount of understanding of how to work with image filters and pyramids. The main diffuculties were to properly build a Laplacian pyramid so that it wouldn't be almost completely black and to filter the time vector.  Unfortunately, I didn't have time to work with all the RGB channels, so the final result is lower in quality than the original video.   I would like to thank Professor Kitani and TA's De-An and Akanksha for helping me and answering my (sometimes not so smart) questions. Also thanks to my classmates from MSIT-SE who helped me with useful advice.