15-869 Image Based Modeling & Rendering

Assignment 2: View Transformation

Results page for:  Chris Rodriguez (cjr2@andrew.cmu.edu)

For this assignment, we used a Minolta Vivid 700 scanner to get a texture and depth range to work with.  Starting with Prof. Heckbert's Inventor format converter code and Prof Seitz's fltk windowing code, I created a program that can read in a Inventor file and display it on the screen.

Here are the images of the object I scanned, a monkey with a banana:

Scene Scanned Depth information retrieved from Monkey (w/ Texture)

From then, I started working on the algorithm from the McMillan Paper.  It was relatively easy to understand and code.  The one problem that I did have was in interpreting the delta values (that they are actually 1/z, not z itself).  This caused a major problem with my interface for a long time - when you rotated the object, it would just fly off the screen!  

Here is a picture of the interface I was using:

I had sliders for the rotation and translation.  I had thought of doing a virtual trackball type thing and doing incremental changes to u, v, o and c and saving them, but it did not seem necessary.  I also have a button for adding in the Z unknown pixels back in, but for the most part it didn't look as good to me.  Also, you can turn hole filling on an off as a speed up.

The way I did the program is with just using OpenGL's raster commands.  I had no real interest in creating a polygon scan converter, and we couldn't use OpenGL.  I used the Grid Data structure from the ivgrid.c file, modified the grid.h to carry the extra data I needed to allow my program to read in Inventor files without outside help.  From there I did a little modification of the mosaic starter code to create my interface, and added a file to do the parts of McMillan's algorithm such as computing the warp as well as the new point of view and visibility determination.  

Here is a picture from before visibility determination and hole filling, but you can clearly see the 3d nature of the point cloud, as well as see some of the missing data::

Finished Product:  Here are some images from the 9 different camera projection positions, as described in the paper.

Here is a 90 image that looks pretty good, but definitely has some issues with hole filling colors, as you can see to the right of his arm where it is a bit lighter color with dark speckles.


My implementation seems to work for the most part, but there are some definite problems. 

you can clearly see the monkey's left arm through his head.  When I turn the hole filling off, you can see through the monkey's face and I believe that's what causes it.


Aside from a few small dumb mistakes, my biggest problem had to be the fact that I was misinterpreting the delta values as Z instead of 1/z which was not allowing me to test the code because I could not rotate.  Once I figured that small fact out, the rest of the project was pretty smooth