Project homepage for 15-869 Image-based Modeling and Rendering

Real-Time Image Mosaicing

Li Zhang, Ph.D. candidate of Robotics

Goal: implementing a coarse-to-fine realtime semi-automatic image mosaicing system with hardware accelerated texture mapping.

Given two images I0 and I1 , we want to find a planar perspective transform M to warp I1 to I0. To make this image registration problem fully automatic is very hard. However, it's is easy to specify an approximate transform M0 in an interactive way, and update M<-(1+D)M automatically.

During this procedure, we need to resample not only color but also derivative at each pixel. This could be accelerated by hardware texture mapping.

If we can use texture mapping to warp colors and derivatives, coase-to-fine registration is a natural application of Mipmap. We only need to specify the width and height of sampling grid, OpenGL will do the resampling.

I have implemented this resampling codes. I used it in my research project, Image-based Mesh Deformation. The resampled color and derivative values could drive an initial mesh to an objective mesh.

The next step would be to set up a user interface, specify initial warping transform, and update it automatically in realtime.

Reference paper:
Image Mosaicing for Tele-Reality Applications, Richard Szeliski
Creating Full View Panoramic Image Mosaic and Enviroment Maps, Richard Szeliski and Heung-Yeung Shum