CMU Advanced Perception Seminar, Spring 1998
Table of Contents
- Class Format
- What Should be in a Critique?
- Grading Policy
- Computer Vision Resources
- Overview of topics, by week
- Week 1. Introduction and Explanation
- Week 2. Feature Extraction I: Edge Extraction
- Week 3. Feature Extraction II: Region Segmentation
- Week 4. Feature Extraction III: Active Contours
- Week 5. Object Recognition I: Model-Based Methods
- Week 6. Object Recognition II: Appearance-Based Methods
- Week 7. Projective Geometry I: Object Indexing Using Invariants
- Week 8. Projective Geometry II: Structure From Multiple Views
- Week 9. Video Analysis I: Stabilization And Mosaicing
- Week 10. Video Analysis II: Feature Tracking And Optical Flow
- Week 11. Video Analysis III: Recovering Egomotion And Structure
- Week 12. New View Synthesis
- Week 13. Human-Computer Interaction
The Advanced Perception course is a graduate reading seminar, meeting once a week to discuss a
set of papers covering a specific topic in computer vision. We will look at historically important
papers in field, as well as current papers from recent conferences and journals. By reading a mixture of both types of papers each week, we will be able to trace the development of some the fundamental ideas that make up current-day computer vision research.
Each week, from 5-7 papers on a particular topic will be assigned. You will read all the papers,
and choose one or more of them as the subject of a short critique (say 2-3 pages) that briefly summarizes and analyzes the paper(s). The critiques will be handed in for grading. During class, the
discussion of each paper will lead off with a short (5-10 minute) presentation by one of the students (one student per paper, assigned the week before) elaborating on their critique of the paper.
This presentation will then evolve into a class discussion lasting anywhere from 20 minutes to an
hour, depending on the paper. The instructors are responsible for keeping the discussion in a fruitful vein and making sure all students get a chance to participate. The instructors are also responsible for making sure that the important points are touched upon during the discussion, which will
sometimes mean asking questions of the class, and for making sure that all the papers are covered
(which sometimes means cutting off discussion and moving on). One practice that has worked
well in the past is to go around the room a few times during the class and get answers from each
person to a specific question, such as `How do you rate this paper on a scale of 1-5?' These questions often spark other discussions and help to involve everyone in the class.
What Should be in a Critique?
Critiques provide a short summary and analysis of the technical content of a paper. Critique writing is an important component of the class, and serves several goals: to give you practice in technical writing, to concretely organize your ideas in preparation for class discussion, and to develop
the skills necessary to become a good conference/journal paper referee. Furthermore, getting in
the habit of writing critiques of the papers you have read will help you do better research - a good
critique provides a concise summary that you can refer to later without having to dig out and read
the original work, and can provide a written starting point for the obligatory literature review section of your own papers/thesis. To help provide you with a sense for what goes into a critique, see
the handout `The Task of the Referee,' by Alan Jay Smith, particularly the section entitled `Evaluating a Research Paper.'
We have found that it is helpful to us, when grading critiques, to have them all in a consistent format. We ask you to hand in critiques with the following sections:
- Citation: the title, author, year, and publication citation of the paper you are reviewing
- Reviewer: your name and the date
- A one-sentence summary (executive abstract) of the paper
- A short overview of the paper including a) key ideas, b) technical approach and c) results.
- Strong points of the paper
- Weak points of the paper
- Questions and Issues
We will grade critiques on a three-level scale: check-minus, check, check-plus. Items 1-6 above
are required to attain a check. In items 7 and 8 we are looking for your resourcefulness, initiative,
creativity and depth of analysis. Doing well in items 7 and 8 will give you a check-plus. Missing
any required sections (1-6) or lack of effort in on of them results in a check-minus.
Pay attention to your spelling and English grammar.
You will be graded on the following items:
|1. Written Critiques||(40%)
|2. Oral Presentations||(20%)
|3. Class Participation||(20%)
|4. Take-Home Final||(20%)
|5. Extra Credit||(10%)
Written critiques form the highest-weighted category, as they represent the bulk of the work that
you will be performing (aside from reading the papers themselves). You will be handing in at least
one critique per week, which will be graded based on your demonstration that you know what the
paper is about and have carefully considered the technical approach and reported results. If you
hand in multiple critiques, the best grade will be recorded and the others will increment your extra
credit account. Extra credit will also be awarded for critiques that compare and contrast two or
more papers that were read that week.
Oral presentation refers to the short summary of a paper that is given in class to jog people's
memory of the paper and lead into the discussion session. It is expected to contain the same information as a critique, but presented orally (using transparencies if you wish). Depending on class
size, you will be giving roughly three oral paper presentations during the semester.
Class participation is rather hard to judge objectively (but we are going to try). We highly encourage you to participate in class discussion, and indeed, this type of class will be a complete failure
if nobody speaks up with their opinions. On the other hand, we don't wish to penalize folks who
aren't naturally talkative. We will try to ensure that even soft-spoken people get a chance to air
their opinions, and will attempt to grade based on the insightfulness of your comments, rather
than the frequency or volume.
There will be a take-home final exam. It will involve writing!
The extra credit category will reflect both objective evidence and subjective impressions we
receive that indicate you are genuinely putting in a lot of effort. Anything you do (of a professional nature, related to this class) that makes us like you better, will increase your extra credit
Computer Vision Resources
There are many places to go to look for computer vision papers, ranging from archival journals to
on-line web sites. Here is a list of our favorite sources of material:
- International Journal of Computer Vision (IJCV)
- Computer Vision and Image Understanding (CVIU)
- used to be Computer Vision, Graphics and Image Processing (CVGIP)
- IEEE Trans on Pattern Analysis and Machine Intelligence (PAMI)
- Image and Vision Computing (IVC)
- Pattern Recognition (PR)
- International Conference on Computer Vision (ICCV)
- Computer Vision and Pattern Recognition (CVPR)
- European Conference on Computer Vision (ECCV)
- DARPA Image Understanding Workshop (IUW)
Overview of Topics by Week
Week 1: Introduction and Explanation
Introduction; explanation of class format and logistics; distribution of papers for week 2. Instructors talk about computer vision resources, and why particular papers were selected for this course.
Discussion of how to write a good critique.
Week 2. Feature Extraction I: Edge Extraction
- E.C.Hildreth, `The Detection of Intensity Changes by Computer and Biological Vision Systems,'
Computer Vision, Graphics and Image Processing, Vol. 27, 1983, pp.1-27.
- J.F.Canny. `A computational approach to edge detection.''
IEEE Trans. on Pattern Analysis
and Machine Intelligence,Vol.8(6), November 1986, pp.679-698.
- J.B.Burns, A.R.Hanson and E.M.Riseman, `Extracting Straight Lines,'
IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.8(4), July 1986, pp. 425-455.
- E.P.Lyvers and O.R.Mitchell, `Precision Edge Contrast and Orientation Estimation,'
Trans on Pattern Analysis and Machine Intelligence, Vol. 10(6), Nov 1988, pp. 927-937.
- M.M.Fleck, `Some Defects in Finite-Difference Edge Finders,'
IEEE Trans on Pattern Analysis and Machine Intelligence, Vol. 14(3), March 1992, pp. 337-345.
- L.R.Williams and D.W.Jacobs, `Stochastic Completion Fields: A Neural Model of Illusory
Contour Shape and Salience,'
Proc.5th International Conference on Computer Vision,
Week 3. Feature Extraction II: Region Segmentation
- J.R.Beveridge, J.S.Griffith, R.R.Kohler, A.R.Hanson and E.M.Riseman, `Segmenting Images
Using Localized Histograms and Region Merging,'
International Journal of Computer
Vision, Vol. 2(3), 1989, pp.311-352.
- L.B.Wolff, `Polarization-Based Material Classification from Specular Reflection,
Pattern Analysis and Machine Intelligence, Vol.12(11), November 1990, pp. 1059-1071.
- R.Bajcsy, S.W.Lee, and A.Leonardis, `Color Image Segmentation with Detection of Highlights and Local Illumination Induced by Inter-reflection,'
Proceedings, 10 Int'l Conference on Pattern Recognition, 1990, pp. 795-90.
- G.Healey, `Segmenting Images using Normalized Color,'
IEEE Trans. on Systems, Man and
Cybernetics, vol. 22(1), 1992, pp. 64-73.
- G.Funka-Lea and R.Bajcsy, `Combining Color and Geometry for the Active, Visual Recognition of Shadows,'
Proc. Int'l Conference on Computer Vision, 1995, pp. 203-209.
- B.Maxwell and S.Shafer, `Physics-Based Segmentation of Complex Objects using Multiple
Hypotheses of Image Formation,'
ComputerVision and Image Understanding, Vol.65(2),
Feb 1997, pp.269-295.
Week 4. Feature Extraction III: Active Contours
- M.Kass, A.Witkin, D.Terzopoulos, `Snakes: Active Contour Models,'
of Computer Vision, Vol.1(4), January 1988, pp. 321-331.
- L.D.Cohen, `On Active Contour Models and Balloons,'
CVGIP: Image Understanding, Vol.
52, no. 2, March 1991, pp. 211-218.
- A.Pentland,and S.Sclaroff, `Closed-Form Solutions for Physically Based Shape Modeling
IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 13, no.
7, July 1991, pp. 715-729.
- M.Isard and A.Blake, `Contour Tracking by Stochastic Propagation of Conditional Density,'
Proc. European Conf. on Computer Vision (best paper award), 1996, pp. 343--356 .
- S.Gunn and M.Nixon, `Robust Snake Implementation: A Dual Active Contour,' IEEE Trans
on Pattern Analysis and Machine Intelligence, Vol. 19(1), January 1997, pp. 63-68.
- T.Cham and R.Cipolla, `Stereo Coupled Active Contours,'
Proc. IEEE Computer Vision and
Pattern Recognition, 1997, pp.1094-1099.
Week 5. Object Recognition I: Model-Based Methods
- D.H.Ballard, `Generalizing the Hough Transform to Detect Arbitrary Shapes,''
Pattern Recognition, Vol.13(2), 1981, pp. 111-122.
- R.Bajcsy and S.Kovacic, `Multiresolution Elastic Matching,'
Computer Vision, Graphics and
Vol 46, 1989, pp.1-21.
- D.P.Huttenlocker and S.Ullman, `Recognizing Solid Objects by Alignment with an Image,'
Int'l Journal of Computer Vision, vol. 5(2), 1990, pp. 195-212.
- W.Grimson, et.al., `An Active Visual Attention System to Play Where's Waldo,'
Image Understanding Workshop,
Monterey CA, Nov. 1994, pp. 1059-1065.
- P.A.Viola and W.Wells, `Alignment by Maximization of Mutual Information,
Journal of Computer Vision, Vol.24(2), September 1997, pp. 137-154.
Week 6. Object Recognition II: Appearance-Based Methods
- H.Murase, and S.K.Nayar, `Visual Learning and Recognition of 3-D Objects from Appearance,'
Int'l Journal of Computer Vision, vol. 14, 1995, pp. 5-24.
- H.Rowley, S.Baluja and T.Kanade, `Human Face Detection in Visual Scenes,' CMU technical report CMU-CS-95-158R, November 1995.
- P.Belhumeur and D.Kriegman, `What is the Set of Images of an Object Under All Possible
Proc. Computer Vision and Pattern Recognition, San Francisco,
CA, June 1996, pp. 270-277.
- F.Idris and S.Panchanathan, `Review of Image and Video Indexing Techniques,'
Visual Communication and Image Representation,
Vol. 8(2), June 1997, pp. 146-166.
The following two papers were packaged together, for the purposes of critiquing:
- M.Flickner et.al., `Query by Image and Video Content: The QBIC System,
No. 9, September 1995, pp. 23-32.
- V.Ogle, `Chabot: Retrieval from a Relational Database of Images',
Vol.28, No. 9,
September 1995, pp. 40-48.
Week 7. Projective Geometry I: Object Indexing Using Invariants
- L.G.Roberts, `Machine Perception of 3-D Solids,'
Optical and Electro-Optical Information
Processing, 1965, pp.159-197.
- D.Forsyth, J.L.Mundy, and A.Zisserman, `Transformational Invariance: A Primer,'
and Vision Computing, Vol.10, 1992, pp. 39-45.
- J.B.Burns, R.S.Weiss and E.M.Riseman, `The Non-Existence of General-Case View-Invariants,'
Geometrical Invariance in Computer Vision, ed. J. Mundy and A.Zisserman, MIT
Press, Cambridge, 1992, pp.120-131.
- Z.Pizlo, A.Rosenfeld and I.Weiss, `The Geometry of Visual Space: About the Incompatibility
Between Science and Mathematics,'
Computer Vision and Image Understanding,
Vol.65(3), March 1997, pp. 425-454. (Includes invited replies, and author's final
The following are background papers on projective geometry. These are not required reading, nor
will they be presented or critiqued.
- R.Duda and P.Hart, Chapter 11: Projective Invariants, in
Pattern Classification and Scene
Wiley, New York, 1973.
- J.L.Mundy and A.Zisserman, `Projective Geometry for Machine Vision,'
Geometrical Invariance in Computer Vision, ed. J. Mundy and A. Zisserman, MIT Press, Cambridge, 1992,
Week 8. Projective Geometry II: Structure From Multiple Views
The following two papers will be treated as one, for the purposes of critiquing:
- H.C.Longuet-Higgins, `A Computer Algorithm for Reconstructing a Scene from Two Projections,'
Nature, vol 293, 1981, pp. 133-135.
- R.Hartley, `In Defense of the 8-point Algorithm,'
IEEE Trans on Pattern Analysis and
Machine Intelligence, 19(6), June 1997, pp. 580-593.
Here are the remaining papers for this week:
- R.Hartley, `Euclidean Reconstruction from Uncalibrated Views,' in
Applications of Invariance in Computer Vision, Edited by Mundy, Zisserman and Forsyth, Springer Verlag,
1994, pp. 237-256.
- A.Shashua and N.Navab, `Relative Affine Structure: Canonical Model for 3D from 2D
Geometry and Applications,'
IEEE Trans on Pattern Analysis and Machine Intelligence,
18(9), September 1996, pp. 873-883. Also MITAI Memo 1489, June 1994.
- C.Rothwell, O.Faugeras and G.Csurka, `A Comparision of Projective Reconstruction Methods for Pairs of Views,'
Computer Vision and Image Understanding, 68(1), Oct 1997,
Week 9. Video Analysis I: Stabilization And Mosaicing
- J.Bergen et.al., `Hierarchical Model-Based Motion Estimation,' in
Proceedings of European
Conference on Computer Vision,
1992, pp. 237-252.
- M.Hansen, P.Anandan, K.Dana, G.van der Wal, and P.Burt, `Real-time Scene Stabilization
and Mosaic Construction,
IEEE Workshop on Applications of Computer Vision, 1994, pp.
- R.Kumar et.al `Representation of Scenes from Collections of Images,'
IEEE Workshop on
Representation of Visual Scenes, June 23, 1995, Cambridge, MA.
- H.Shum and R.Szeliski, `Construction and Refinement of Panoramic Mosaics with Glocal
and Local Alignment,'
International Conference on Computer Vision, Bombay, India,
Jan.1998, pp. 953-958.
- B.Rousso, S.Peleg, I.Finci and A.Rav-Acha, `Universal Mosaicing using Pipe Projection,'
International Conference on Computer Vision, Bombay, India, Jan.1998, pp. 945-952.
Week 10. Video Analysis II: Feature Tracking And Optical Flow
- B.D.Lucas and T.Kanade, `An Iterative Image Registration Technique with an Application in
Stereo Vision,' In
Proceedings of Seventh International Joint Conference on Artificial
Intelligence (IJCAI), 1981, pp. 674-679.
- J.Wang and E.Adelson, `Representing Moving Images with Layers,'
IEEE Trans on Image
Processing Special Issue: Image Sequence Compression, Vol.3(5), September 1994,
- J.L.Barron, D.J.Fleet, and S.S.Beauchemin, `Performance of Optical Flow Techniques,'
Journal of Computer Vision, vol. 12, no. 1, Jan. 1994, pp. 43-77.
- D.Koller, K.Daniilidis, and H.Nagel, `Model-Based Object Tracking in Monocular Image
Sequences of Road Traffic Scenes,
International Journal of Computer Vision, Vol.10(3),
June 1993, pp. 257-281.
- G.Halevi and D.Weinshall, `Motion of Disturbances: Detection and Tracking of Multi-Body
Conf. Computer Vision and Pattern Recognition, 1997, pp.897-902.
Week 11. Video Analysis III: Recovering Egomotion And Structure
- B.K.P.Horn and W.J.Weldon, Jr., `Direct Methods of Recovering Motion,'
Journal of Computer Vision, vol. 2, 1988, pp. 51-76.
- A.Azarbayejani and A.Pentland, `Recursive Estimation of Motion, Structure, and Focal
IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 17, no. 6, June
1995, pp. 562-575.
- P.Torr, A.Fitzgibbon and A.Zisserman, `Maintaining Multiple Motion Model Hypotheses
over Many Views to Recover Matching and Structure,'
International Conference on Computer Vision,
Bombay, 1998, pp.485-491.
- C.Tomasi and T.Kanade, `Shape and Motion from Image Streams under Orthography: a Factorization Method,'
International Journal of Computer Vision, Vol. 9(2), 1992, pp. 137-154.
- D.Morris and T.Kanade, `A Unified Factorization Algorithm for Points, Line Segments and
Planes with Uncertainty Models,
International Conference on Computer Vision,
Week 12. New View Synthesis
- S.E.Chen and L.Williams, `View Interpolation for Image Synthesis,'
- L.McMillan and G.Bishop, `Plenoptic Modeling: An Image-Based Rendering System,'
- S.Gortler, R.Grzeszczuk, R.Szeliski and M.Cohen, `The Lumigraph,'
- S.Seitz and C.Dyer, `View Morphing,'
- P.Rander, P.Narayanan and T.Kanade, `Virtualized Reality,'
IEEE Visualization 97, Phoenix
Arizona, Oct.19-23, 1997, pp.277-283.
Week 13. Human-Computer Interaction
- F.Quek, T.Mysliwiec, and M.Zhao, `FingerMouse: A Freehand Pointing Interface,' in
Proceedings of Int'l Workshop on Automatic Face and Gesture Recognition, 1995, pp. 372-377.
- M.J.Black and Y.Yacoob, `Tracking and Recognizing Facial Expressions in Image Sequences
using Local parameterized Models of Image Motion,' Technical Report, University of
Maryland Computer Science Dept., CS-TR-3401, January 1995.
- T.Darrell and A.Pentland, `Space-Time Gestures,' in
Proceedings of IEEE Conference on
Computer Vision and Pattern Recognition,
- F.Quek, `Eyes in the Interface,'
Image and Vision Computing, Vol. 13, no. 6, August 1995,
- J.M.Rehg, M.Loughlin and K.Waters, `Vision for a Smart Kiosk,'
IEEE Conf Computer
Vision and Pattern Recognition, 1997, pp. 690-696.
- A.Bobick et.al., `The Kidsroom: A Perceptually-Based Interactive and Immersive Story Environment,' MIT Media Lab Perceptual Computing Section Technical Report No. 398, September 1997.