15-869 IMAGE BASED MODELING AND RENDERING
ASSIGNMENT - 1 IMAGE MOSAICING
Kiran Bhat : kiranb@ri
For the assignment, I used a video sequence taken from a camera mounted on a pan-tilt unit. This sequence was recorded inside my office. The sequence was digitized by an MRT framegrabber, and the video sequence was directly mosaicked in pseudo- real time (2-3Hz). The registration is fully automatic, with corresponding points being generated by a feature tracker. Figure 1 shows 6 snapshots of the input video sequence (camera pan to the right).
1 & 2
3 & 4
5 & 6
Figure 1: Snapshots of the input sequence
(sequence: Left Right; Right left)
Figure 2: Feature tracker results (between frame 1 & 2 of the input sequence)
Figure 2 shows the results of the feature tracker, that automatically locates and tracks features between successive frames. I use the techniques developed by Tomasi and Kanade (the KLT feature tracker) for robust feature tracking. Here, an image is divided into small blocks of size 8*8 pixels. There are two basic goals: one, to identify good feature points, and two, to track them robustly (find the translation dx and dy of each point in successive frames). This is done by looking at the eigenvalues of the hessian matrix obtained wrt the translation parameters. If the eigenvalues are larger than a particular threshold, the feature points are selected and tracked. I found that this technique is pretty robust and fast (works at around 3-5 Hz).
From the feature tracker, I obtain the (x,y) coordinates of around 80 feature points between each images of the input sequence.( I initialise around 100 feature points, but some of them are lost during the camera motion). I compute an affine transformation between the images by solving for an overconstrained system:
ie, for n feature points,
[ x’ y’ 1]’ = [a b e ; c d f ; 0 0 1] * [x y 1]’
After warping the input sequence using the above mentioned transformation, the following mosiac is obtained.
Highlights of this method:
- Works at 2-3Hz with a pentium-150 laptop, with 64Mb RAM.
Future improvements:
- Color Mosaicing