Overview
In this project, I implemented Poisson Blending (Perez et al., 2003) The goal is to seamlessly blend an object or texture from a source image into a target image. The user can select a region from source image containing an object he wants to blend, and specify a location in background image at which the object will be blended. With Poisson Blending, we find values for the target pixels that maximally preserve the gradient of the source region.
Approach
In order to find the best pixel intensities in the target region, I created variables "v_i" that represents the target intensity at i-th pixel. Given the pixel intensities of the source image "s" and of the target image "t", I solved for new intensity values "v" within the source region "S" by minimizing the following objective function:
In Matlab, I converted this objective function into a set of least squares constraints in the standard matrix form: (Av-b)^2, where "A" is a sparse matrix, "v" are the variables to be solved, and "b" is a known vector. To solve for v, I used v = lscov(A, b). I did this process for each channel independently and the results from 3 channels are combined into the output image.
Results
Toy Problem
First, I used toy image to test the correctness of the code for basic gradient-domain processing. I computed the x and y gradients from an image s, then use all the gradients, plus one pixel intensity, to reconstruct an image v.
Original |
Reconstructed |
 |
 |
The sum of squared error between the original image and the reconstructed image is very low (2.714e-6), showing that the algorithm works perfectly.
Poisson Blending
I ran Poisson Blending on a variety of background images and objects (photographs, drawings, texts). In most cases, the algorithm is able to blend the object in seamlessly, although when there is a high difference in color and lighting between the object and destination region, the object's color can be radically altered and look very unnatural.
Background + Object |
Result |
 Snow Moutain Hiking  South Pole Penguin Chick |
 Snow Mountain Penguin |
 Countryside Illustration  Fighter Aircraft |
 The aircraft blends in seamlessly, even though its whole body turns bluish |
 The Iron Throne  Daenerys Stormborn |
 Daenerys Targaryen. Queen of the Andals, the Rhoynar and the First Men, Lord of the Seven Kingdoms, Protector of the Realm |
 Occupied Iron Throne Daenerys Stormborn |
 The source adjusts well to a different lighting, although the discontinuity around the edges is more apparent in this one. |
 Gurren Lagann Mecha  Cool Japanese text |
 "Final Episode: All the Lights in the Sky are Stars" With Poisson Blending, the area around the text looks blurry. |
Mixed Gradients
To fix the blurry background, I used the same methods as Poisson blending, but use the gradient in source or target (whichever has a larger magnitude) as the guide, instead of just the source gradient. As shown below, with mixed gradients the blurriness is completely removed, but some parts of the text can become transparent.
Poisson Blending |
Mixed Gradients |
|
 |
Bells and Whistles
Color2Gray
This is another application of gradient domain processing. I implemented a function for converting color image to grayscale that preserves constrast information, unlike Matlab's rgb2gray(). The objective function for this task is:
where "g_i" is the V value of the i-th pixel. d_ij is the gradient in the S and V channel. The parameter "beta" is for adjusting the influence of the first term. To make the following image, I set beta to 5.
Original |
rgb2gray |
Gradient Domain (beta=5) |
 |
 |
 |