Computational Perception: 15-485/785, Spring 2008

Instructor:

Mike Lewicki  (lewicki@cs)

Mellon Institute 115K, x8-3921

Office hours by appointment


Lectures:

Tue/Thu 10:30 - 11:50, Porter Hall A19 


Description:

How brains do it? The perceptual capabilities of even the simplest biological organisms are far beyond what we can achieve with machines. Whether you look at sensitivity, robustness, or sheer perceptual power, perception in biology just works, and works in complex, ever changing environments, and can pick up the most subtle sensory patterns. Is it the neural hardware? Does biology solve funamentally different problems? What can we learn from biological systems and human perception? 


This course teaches advanced aspects of perception and scene analysis in both the visual and auditory modalities, concentrating on those aspects that allow us and animals to behave in natural, complex environments. In this course, you will learn how to reason scientifically about problems and issues in perception and scene analysis, how to extract the essential computational properties of those abstract ideas, and finally how to convert these into explicit mathematical models and computational algorithms. In the process, you will cover a wide range of literature that provide a very different perspectives on problems and properties of natural perception.


Course information:

• syllabus

• class topics, readings, and references


Schedule:

This is subject to change and is updated here.


Date

Topics

Handouts

Assignments & Notes

1/15

Course overview and general issues [slides]
  • Nakayama, K. (1998) Vision fin-de-siecle - a reductionistic explanation of perception for the 21st century? [pdf]
  • von Bekesy, G. (1960) The problems of auditory research. [pdf]

Read both of these overview articles on perception science. 

1/17

Sound localization: Part 1 [slides]
  • Moore, B. C. J. (1997)
    • Ch.1: The nature of sound and the structure and function of the auditory system
    • Ch.6: Space perception
  • Yost, W. A. (2000)
    • Ch.3: Sound transmission
  • Castleman, K. R. (1996)
    • Ch.9: Linear systems theory
    • Ch.10: The Fourier transform

These handouts are provided for background.  Use them as reference material for the auditory lectures and homework assignments.

1/22

Part 2 [slides]
  • Kulkarni & Colburn (1998) Role of spectral detail in sound-source localization.
  • Semple (1998) Auditory perception - sound in a virtual world. (News and Views article of Kulkarni & Colburn)

Read the handout for class discussion on Thursday 1/24.


HW1 out. [pdf] [materials]


For background and to get started with matlab you can go through the tutorial on filtering, convolution, and the Fourier domain using MATLAB. [pdf] [code]

1/24

Part 3  [slides]



1/29

Bayesian inference and modeling [slides] 


 A tutorial on classical and Bayesian inference [pdf] 

1/31

Part 2 (on board)


HW1 due; HW2 out.  [pdf] [materials]

2/5

Auditory coding : Part 1  [slides] 
  •  Smith and Lewicki (2006) Efficient Auditory Coding.

 

2/7

Part 2 [slides] 



2/12

Visual coding: Part 1  [slides] 



2/14

NO CLASS


 HW2 due; HW3 out.  [pdf] [materials]

2/19

Part 2  [slides] 

 

 

2/21

Part 3  [slides] 

 

 

2/26

Visual motion: Part 1  [slides] 
  • Adelson and Bergen (1985) Spatiotemproral energy models for the perception of motion.
  • Weiss, Simoncelli, and Adelson (2002) Motion illusions as optimal percepts.

HW3 due; HW4 out.   [pdf] [materials]

2/28

Part 2 [slides] 

 

 

3/4

Perceptual inference: Part 1 [slides] 

 

 

3/6

Part 2 [slides] 

 

HW4 due

3/10-14

(no class - spring break)

 

 

3/18

Visual structure, representations of shape and surfaces: Part 1 [slides]



3/20

Part 1 (cont'd)
  • Fleming, Dror, and Adelson (2003) Real-world illumination and the perception of surface reflectance properties. [pdf]
  • Fleming, Torralba, and Adelson (2004) Specular reflections and the perception of shape. [pdf]

 

3/25

(no class)

 


3/27

Visual structure: Part 2 [slides]

 

 grad project proposals due

4/1

Perceptual constancy: Part 1 [slides]


 

4/3

Parts 2a,2b  [slides, slides]


 HW5 [pdf] [materials] 

4/8

Auditory structure Part 1 [slides]

 

 

4/10

Part 2 [slides]

 

 

4/15

Auditory scene analysis [slides]

 


4/17

(no class; spring carnival)

 

 

4/22

Eye movements, visual search, & attention [slides]

 

HW5 due 

4/24

Visual scene analysis

 

 

4/29

Perceptual organization

 

 

5/1

Project presentations

 

project presentations due

5/2

 

 

grad projects due