VASC Seminar Announcement ========================= Date: Monday, 5/17/99 Time: 3:30-4:30 Place: Smith Hall 2nd Floor Common Area Speaker: James Crowley National Polytechnic Institute-Grenoble, France http://pandora.imag.fr/Prima/jlc.html Title: The appearance manifold as a foundation for computer vision Abstract: The images of an object deform continuously with continuous changes in view direction, illumination and camera parameters. When images are discretely sampled, each image can be seen as a vector in a very high dimensional space. The continuous space of all possible images of an object and its background forms a relatively low dimensional manifold embedded in this high dimensional space, as demonstrated by Murase and Nayar (1995). The dimensionality of the appearance manifold is determined by the number of degrees of freedom in viewing conditions, including view direction, illumination and camera parameters. In order to use the appearance manifold as an approach for view invariant object recognition, it is necessary to overcome a number of difficult problems. One of the most intractable problems is dependence of the appearance manifold on the object background. A related problem is sensitivity of the manifold to occlusions. The common approach to avoid problems with background and occlusion is to segment out the region of the image which corresponds to the object. This is the approach which was adopted for face recognition by Turk and Pentland (1991) and for object recognition by Murase and Nayar. Unfortunately, such image segmentation is a classic hard problem for which there does not appear to be a general solution. The effects of background and occlusion can be avoided by expressing the appearance manifold in a local appearance space. A local appearance space is defined by projecting local neighborhoods (imagettes) into a linear subspace, thereby obtaining a discretely sampled surface. Each sample of this surface corresponds to the projection of an imagette. Projecting overlapping imagettes gives a dense grid (or mesh) of samples. Regions of the image which correspond to background are expressed as local regions of the mesh which can be ignored. The effects of occlusion are similarly isolated in local regions of the mesh. This talk describes recent results in the investigation of techniques for sampling, representing and matching the local appearance manifold of objects. hese techniques lead to a robust method for visual recognition which exhibits a very low computational cost. We describe experiments which compare the use of principal components analysis of small windows (imagettes) with Gaussian derivative filters. We show how to obtain invariance to scale by normalising using a Laplacian scale space. We show how to obtain invariance to image plane rotation using normalisation based on local orientation computed with steerable filters. We show how to obtain invariance to illumination intensity using energy normalisation. Recognition is achieved by projecting small neighborhoods from a newly acquired image into the local appearance space and associating them to nearby surfaces. An efficient tree based search technique is used to associate the projection of windows to surfaces. Our results show that in many common situations, a single window is sufficient to determine the correct object, viewing angle and image neighborhood. Robust recognition is obtained by using the mutual spatial coherence provided by multiple windows in a two phases recognition algorithm based on prediction and verification. We conclude with applications of the technique to problems of mobile robot localisation, dense stereo matching, video compression, recognition of gestures, facial expressions and activities. Recent Papers concerning the talk: 1) V. Colin de Verdière and J. L. Crowley, "A Prediction-Verification Strategy for Object Recognition using Local Appearance", Tech report, Project PRIMA, INRIA Rhône-Alpes, April 1999. 2) O. Chomat and J. L. Crowley, "Probabilistic Recognition of Activity using Local Appearance", IEEE Conference on Computer Vision and Pattern Recognition, CVPR 99, Fort Collins, June 1999. 3) W. Vieux, K. Schwerdt, and J. L. Crowley, "Face-tracking and coding for video compression", ICVS 99, International Conference on Vision Systems, Springer LNCS Series, Jan 1999. 4) F. Pourraz and J. L. Crowley, "Continuity Properties of the Appearance Manifold for Mobile Robot Position Estimation", Symposium on Intelligent Robotics Systems, SIRS-98, Edinburgh, July 1998. 5) V. Colin de Verdiere and J. L. Crowley, "Visual Recognition Using Local Appearance", ECCV '98, Frieburg, June, 1998. 6) J. L. Crowley, F. Wallner and B. Schiele, "Position Estimation Using Principal Components of Range Data", 1998 IEEE Conference on Robotics and Automation, Leuven, May, 1998. 7) B. Schiele and J. L. Crowley, "Transinformation for Active Object Recognition", In ICCV'98, International Conference on Computer Vision, Bombay, India, January 1998 8) B. Schiele and J.L. Crowley, "Probabilistic Object Recognition Using Multidimensional Receptive Field Histograms", ICPR '96, Vienna, August 96.