Computational Perception and Scene Analysis

Tue/Thu 10:30-11:50, Porter Hall A19

Spring, 2006

Note: This course will be offered in Spring, 2008. This web page is from Spring, 2006 course and will be updated.

Description:

How brains do it? The perceptual capabilities of even the simplest biological organisms are far beyond what we can achieve with machines. Whether you look at sensitivity, robustness, or sheer perceptual power, perception in biology just works, and works in complex, ever changing environments, and can pick up the most subtle sensory patterns. Is it the neural hardware? Does biology solve funamentally different problems? What can we learn from biological systems and human perception?

This course teaches advanced aspects of perception and scene analysis in both the visual and auditory modalities, concentrating on those aspects that allow us and animals to behave in natural, complex environments. In this course, you will learn how to reason scientifically about problems and issues in perception and scene analysis, how to extract the essential computational properties of those abstract ideas, and finally how to convert these into explicit mathematical models and computational algorithms. In the process, you will cover a wide range of literature that provide a very different perspectives on problems and properties of natural perception.

Instructor:

Mike Lewicki
email: lewicki at cs dot cmu dot edu,   office: Mellon Institute 115K,   phone: 268.3921
Office hours: by appointment

Teaching Assistant:

Eizaburo Doi
email: edoi at cnbc dot cmu dot edu,   office: Mellon Institute 116B,   phone: 268.1710
Office hours: Mon 13:00-14:30 or by appointment

Lecture schedule: (syllabus, class topics, readings, and references)

The schedule is subject to change and is updated here.

Date Topics Handout Announcement
1/17 Course overview and general issues [slides]
  • Nakayama, K. (1998) Vision fin-de-siecle - a reductionistic explanation of perception for the 21st century? [pdf]
  • von Bekesy, G. (1960) The problems of auditory research.
 
1/19 Sound localization, linear systems theory, and Bayesian inference: Part 1. [slides]
  • Moore, B. C. J. (1997)
    • Ch.1: The nature of sound and the structure and function of the auditory system
    • Ch.6: Space perception
  • Yost, W. A. (2000)
    • Ch.3: Sound transmission
  • Castleman, K. R. (1996)
    • Ch.9: Linear systems theory
    • Ch.10: The Fourier transform
 
1/24 Part 2. [slides]
  • A tutorial on filtering, convolution, and the Fourier domain using MATLAB. [pdf] [code]
HW1 out [pdf] [materials]
1/26 Part 3. [slides]
  • King, Schnupp, & Doubell (2001) The shape of ears to come: dynamic coding of auditory space.
  • Kulkarni & Colburn (1998) Role of spectral detail in sound-source localization.
  • Semple (1998) Auditory perception - sound in a virtual world. (News and Views article of Kulkarni & Colburn)
 
1/31 Bayesian inference [slides]
  • Lewicki (2002) Efficient coding of natural sounds.
  • Olshausen & O'Connor (2002) A new window on sound. (News and Views article of Lewicki)
 
2/2 Auditory coding : Part 1. [slides]
  • A tutorial on classical and Bayesian inference [pdf]
HW1 DUE
HW2 out [pdf] [materials]
2/7 Part 2. [slides]    
2/9 Visual coding: Part 1. [slides] HW1 returned
2/14 Visual coding: Part 2. [slides]
    • Ch.5: The retinal representation
    • Ch.8: Multiresolution image representations
HW2 DUE
HW3 out [pdf] [materials]
2/16 Visual coding: Part 3. [slides]
  • Atick (1992) Could information theory provide an ecological theory of sensory processing?
  • Lewicki & Olshausen (1999) Probabilistic framework for the adaptation and comparison of image codes.
  • Olshausen & Field (2000) Vision and the coding of natural images.
 
2/21 Visual coding: Part 4. [slides]    
2/23 Mike sick - class canceled.    
2/28 Computation and representation of visual motion and regularization: Part 1. [slides]
[demos (pdf)] [demos (quicktime)]
  • Wandell, B. A. (1995) Foundation of Vision
    • Ch.10: Motion and Depth
  • Horn, B. K. (1986) Robot Vision.
    • Ch.10: Motion Field and Optical Flow
  • Weiss, Simoncelli, & Adelson (2002) Motion illusions ans optimal percepts. [pdf]
HW3 DUE
HW4 out [pdf] [materials]
3/2 Part 2. [slides]
    
3/7 Perceptual inference and Bayesian modeling: Part 1. Lecturer: Yan Karklin [slides]
    
3/9 Part 2. Lecturer: Yan Karklin [slides]
    
3/13-17 (no class; spring break)    
3/21 Visual structure, representations of shape and surfaces: Part 1 [slides]
   HW2 returned
3/23 Part 2 [slides]
    
3/28 Perceptual constancy: Part 1 [slides]
   HW3 returned
HW5 out [pdf]
3/30 Part 2a [slides] Part 2b [slides]
    
4/4 Auditory structure Part 1 [slides]
    
4/6 Part 2 [slides]
    
4/11 Auditory scene analysis: Part 1. [slides]
Heather's presentation 		    HW5 DUE
4/13 Part 2 [slides]
Jason's presentation 		   
4/18 Eye movements [slides]
Yi's presentation 		 
4/20 (no class; spring carnival)    
4/25 Eye movements and visual search [slides]
     HW6 out [pdf] [materials]
4/27 Visual scene analysis [slides]
    
5/2 Perceptual organization [slides]
    
5/4 Review and retrospective   HW6 DUE
5/5     PROJECT DUE
Michael S. Lewicki
Last modified: Tue Apr 18 22:06:43 EDT 2006