Syllabus for 15-869, Image-Based Modeling and Rendering

• Introduction
  • what are image-based techniques?
  • administrative matters
  • video examples
• Images (2D)
  • image warping
    • transformations: rigid body, affine, projective, other
    • brief review of homogeneous coordinates
    • resampling, bilinear interpolation
    • algorithms for image warping: forward, backward, two-pass
  • image mosaics
    • applications: remote sensing of earth and planets, panoramas, superresolution, video
    • projective mosaics: planar scenes and panoramas
    • environment maps: cylinder, sphere, cube
    • Quicktime VR
  • image registration for mosaicing
    • manual methods
    • automatic methods: sum of squared differences, mutual information
  • image compositing
    • alpha blending
    • feathering for image mosaics
  • morphing
    • correspondence specification
    • Beier-Neely method
    • grid-based techniques
  • interactive techniques
    • finding image contours with "snakes"
    • matte extraction, "scissors"
    • environment matting
• Depth Images (2.5-D)
  • z-buffer, range data, disparity
  • view interpolation
    • review of perspective camera model
    • Chen-Williams algorithm
    • occlusion, hole filling problem
    • plane + parallax
    • Macmillan's back-to-front rendering algorithm, plenoptic modeling
  • sprites (images in layers)
    • automatic extraction of image layers from video
    • layered depth images (LDI's)
    • the (proposed) Talisman rendering architecture
    • impostors (resampling geometric models)
• 3D Modeling
  • cameras
    • optics of real cameras
    • projective geometry (point-line duality)
    • view morphing (Seitz)
  • representing geometry: meshes, voxels, point clouds
  • texture mapping
    • applications: reflectance, bumps, transparency, specularity, environment
    • antialiasing and texture filtering
  • acquiring geometry
    • camera calibration
    • photogrammetry
      • Debevec's FACADE system
    • stereo
      • correspondence problem, epipolar geometry
      • stereo algorithms, occlusion, hole filling problem
    • structure from motion -maybe
    • voxel and silhouette models
      • volume intersection
      • finding silhouettes (again)
      • voxel coloring (Seitz & Dyer)
    • active methods -maybe
    • merging range scans -maybe
  • representing radiance
    • the many-dimensional nature of radiance
    • typical limitation to diffuse, opaque surfaces
    • bi-directional reflectance distribution functions (BRDF's)
    • environment maps
  • computing radiance/BRDF's
    • ray tracing
    • high dynamic range images
    • compositing real and synthetic images
• Ray-based representations (4D)
  • light fields: acquiring, compression, rendering (Levoy, Gortler)
  • Shum's concentric mosaics
• Applications
  • Faces
    • model-based techniques: Blanz & Vetter
    • learning techniques
  • special effects: Dayton Taylor's "timetrack"
  • teleconferencing -maybe
  • image and video rewrite
• Motion -maybe
  • optical flow

Steve Seitz and Paul Heckbert, Sept. 1999