Image-based Shaving

Minh Hoai Nguyen, Jean-François Lalonde, Alexei A. Efros and Fernando de la Torre

Abstract

Many categories of objects, such as human faces, can be naturally viewed as a composition of several different layers. For example, a bearded face with glasses can be decomposed into three layers: a layer for glasses, a layer for the beard and a layer for other permanent facial features. While modeling such a face with a linear subspace model could be very difficult, layer separation allows for easy modeling and modification of some certain structures while leaving others unchanged. In this paper, we present a method for automatic layer extraction and its applications to face synthesis and editing. Layers are automatically extracted by utilizing the differences between subspaces and modeled separately. We show that our method can be used for tasks such beard removal (virtual shaving), beard synthesis, and beard transfer, among others.

Citation

Minh Hoai Nguyen, Jean-François Lalonde, Alexei A. Efros and Fernando de la Torre Image-based Shaving. Computer Graphics Forum Journal (Eurographics 2008), 27(2): 627-635, 2008 [Project Page]

Photo Clip Art

Jean-François Lalonde, Derek Hoiem, Alexei A. Efros, Carsten Rother, John Winn and Antonio Criminisi

Abstract

We present a system for inserting new objects into existing photographs by querying a vast image-based object library, precomputed using a publicly available Internet object database. The central goal is to shield the user from all of the arduous tasks typically involved in image compositing. The user is only asked to do two simple things: 1) pick a 3D location in the scene to place a new object; 2) select an object to insert using a hierarchical menu. We pose the problem of object insertion as a data-driven, 3D-based, context-sensitive object retrieval task. Instead of trying to manipulate the object to change its orientation, color distribution, etc. to fit the new image, we simply retrieve an object of a specified class that has all the required properties (camera pose, lighting, resolution, etc) from our large object library. We present new automatic algorithms for improving object segmentation and blending, estimating true 3D object size and orientation, and estimating scene lighting conditions. We also present an intuitive user interface that makes object insertion fast and simple even for the artistically challenged.

Citation

Jean-François Lalonde, Derek Hoiem, Alexei A. Efros, Carsten Rother, John Winn and Antonio Criminisi Photo Clip Art. ACM Transactions on Graphics (SIGGRAPH 2007), August 2007, Vol. 26, No. 3 [Project Page]

Data Structure for Efficient Dynamic Processing in 3-D

Jean-François Lalonde, Nicolas Vandapel, and Martial Hebert

Abstract

In this paper, we consider the problem of the dynamic processing of large amounts of sparse three-dimensional data. It is assumed that computations are performed in a neighborhood defined around each point in order to retrieve local properties. This general kind of processing can be applied to a wide variety of problems. We propose a new, efficient data structure and corresponding algorithms that significantly improve the speed of the range search operation and that are suitable for on-line operation where data is accumulated dynamically. The method relies on taking advantage of overlapping neighborhoods and the reuse of previously computed data as the algorithm scans each data point. To demonstrate the dynamic capabilities of the data structure, we use data obtained from a laser radar mounted on a ground mobile robot operating in complex, outdoor environments. We show that this approach considerably improves the speed of an established 3-D perception processing algorithm.

Citation

Jean-François Lalonde, Nicolas Vandapel and Martial Hebert. Data Structure for Efficient Dynamic Processing in 3-D. International Journal of Robotics Research, 26(8):777-796, August 2007 [PDF], [BibTeX]

Conference version

Jean-François Lalonde, Nicolas Vandapel and Martial Hebert. Data Structure for Efficient Processing in 3-D. Robotics: Science and Systems I, 2005. [PDF], [BibTeX]

Technical Report (Master's Thesis)

Jean-François Lalonde. Data Structure for Efficient Dynamic Processing in 3-D. Master's thesis, technical report CMU-RI-TR-06-22, Robotics Institute, Carnegie Mellon University, May, 2006. [PDF], [BibTeX]

Natural Terrain Classification using Three-Dimensional Ladar Data for Ground Robot Mobility

Jean-François Lalonde, Nicolas Vandapel, Daniel F. Huber and Martial Hebert

Abstract

In recent years, much progress has been made in outdoor autonomous navigation. However, safe navigation is still a daunting challenge in terrain containing vegetation. In this paper, we focus on the segmentation of ladar data into three classes using local threedimensional point cloud statistics. The classes are: "scatter" to represent porous volumes such as grass and tree canopy; "linear" to capture thin objects like wires or tree branches, and finally "surface" to capture solid objects like ground surface, rocks, or large trunks. We present the details of the proposed method, and the modifications we made to implement it on-board an autonomous ground vehicle for real-time data processing. Finally, we present results produced from different stationary laser sensors and from field tests using an unmanned ground vehicle.

Citation

Jean-François Lalonde, Nicolas Vandapel, Daniel F. Huber and Martial Hebert. Natural Terrain Classification using Three-Dimensional Ladar Data for Ground Robot Mobility. Journal of Field Robotics, 23(10):839--861, October 2006. [PDF], [BibTeX]

Using Color Compatibility for Assessing Image Realism

Jean-François Lalonde and Alexei A. Efros.

Abstract

Why does placing an object from one photograph into another often make the colors of that object suddenly look wrong? One possibility is that humans prefer distributions of colors that are often found in nature; that is, we find pleasing these color combinations that we see often. Another possibility is that humans simply prefer colors to be consistent within an image, regardless of what they are. In this paper, we explore some of these issues by studying the color statistics of a large dataset of natural images, and by looking at differences in color distribution in realistic and unrealistic images. We apply our findings to two problems: 1) classifying composite images into realistic vs. non-realistic, and 2) recoloring image regions for realistic compositing.

Citation

Jean-François Lalonde and Alexei A. Efros. Using Color Compatibility for Assessing Image Realism. International Conference on Computer Vision, 2007. [Project Page]

Potential negative obstacle detection by occlusion labeling

Nicholas Heckman, Jean-François Lalonde, Nicolas Vandapel and Martial Hebert

Abstract

In this paper, we present an approach for potential negative obstacle detection, based on missing data interpreta- tion that extends traditional techniques driven by data only, which capture the occupancy of the scene. The approach is decomposed into three steps: three-dimensional (3-D) data accu- mulation and low level classification, 3-D occluder propagation, and context-based occlusion labeling. The approach is validated using logged laser data collected in various outdoor natural terrains and also demonstrated live on-board the Demo-III eXperimental Unmanned Vehicle (XUV).

Citation

Nicholas Heckman, Jean-François Lalonde, Nicolas Vandapel and Martial Hebert Potential negative obstacle detection by occlusion labeling. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007. [PDF], [BibTeX]

Scale Selection for the Analysis of Point-Sampled Curves

Ranjith Unnikrishnan, Jean-François Lalonde, Nicolas Vandapel and Martial Hebert.

Abstract

An important task in the analysis and reconstruction of curvilinear structures from unorganized 3-D point samples is the estimation of tangent information at each data point. Its main challenges are in (1) the selection of an appropriate scale of analysis to accomodate noise, density variation and sparsity in data, and in (2) the formulation of a model and associated objective function that correctly expresses their effects. We pose this problem as one of estimating the neighborhood size for which the principal eigenvector of the data scatter matrix is best aligned with the true tangent of the curve, in a probabilistic sense. We analyze the perturbation on the direction of the eigenvector due to finite samples and noise using the expected statistics of the scatter matrix estimators, and employ a simple iterative procedure to choose the optimal neighborhood size. Experiments on synthetic and real data validate the behavior predicted by the model, and show competitive performance and improved stability over leading polynomial-fitting alternatives that require a preset scale.

Citation

Ranjith Unnikrishnan, Jean-François Lalonde, Nicolas Vandapel and Martial Hebert. Scale Selection for the Analysis of Point-Sampled Curves. Third International Symposium on 3D Processing, Visualization and Transmission (3DPVT 2006), 2006. [PDF], [BibTeX]

Technical report

Ranjith Unnikrishnan, Jean-François Lalonde, Nicolas Vandapel and Martial Hebert. Scale Selection for the Analysis of Point-Sampled Curves: Extended report. tech. report CMU-RI-TR-06-25, Robotics Institute, Carnegie Mellon University, June, 2006. [PDF], [BibTeX]

Mobile Robot Programming in Education

Jean-François Lalonde, Christopher P. Bartley and Illah Nourbakhsh

Abstract

The Mobile Robot Programming course has been taught at Carnegie Mellon University for the past twelve years. It is a problem-driven class designed for students with little or no experience with robots. In this paper, we first present the current status of the class, and show how it improves the education and training of students in a robotics curriculum by giving them a hands-on experience with a real robot. We show that, in addition to core subjects such as perception, action and cognition, students also have the opportunity to learn advanced topics such as reinforcement learning and multi-robot coordination. We then discuss the evolution of the class under general categories: hardware and programming environment, team experiments, and assignments. We present important lessons learned in each category, and how they affect the learning experience of participating students. We conclude by discussing future opportunities.

Citation

Jean-François Lalonde, Christopher P. Bartley and Illah Nourbakhsh. Mobile Robot Programming in Education. International Conference on Robotics and Automation, 2006. [PDF], [BibTeX]

Scale Selection for Classification of Point-sampled 3-D Surfaces

Jean-François Lalonde, Ranjith Unnikrishnan, Nicolas Vandapel and Martial Hebert

Abstract

Three-dimensional ladar data are commonly used to perform scene understanding for outdoor mobile robots, specifically in natural terrain. One effective method is to classify points using features based on local point cloud distribution into surfaces, linear structures or clutter volumes. But the local features are computed using 3-D points within a support-volume. Local and global point density variations and the presence of multiple manifolds make the problem of selecting the size of this support volume, or scale, challenging. In this paper we adopt an approach inspired by recent developments in computational geometry and investigate the problem of automatic data-driven scale selection to improve point cloud classification. The approach is validated with results using data from different sensors in various environments classified into different terrain types (vegetation, solid surface and linear structure).

Citation

Jean-François Lalonde, Ranjith Unnikrishnan, Nicolas Vandapel and Martial Hebert. Scale Selection for Classification of Point-sampled 3-D Surfaces. Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM), June 2005. [PDF], [BibTeX]

Technical report

Jean-François Lalonde, Ranjith Unnikrishnan, Nicolas Vandapel and Martial Hebert. Scale Selection for the Analysis of Point-Sampled Curves: Extended report. tech. report CMU-RI-TR-05-01, Robotics Institute, Carnegie Mellon University, June, 2005. [PDF], [BibTeX]

Projector-Based Dual-Resolution Stereoscopic Display

Guy Godin, Jean-François Lalonde and Louis Borgeat

Abstract

We present a stereoscopic display system which incorporates a high-resolution inset image, or fovea. We describe the specific problem of false depth cues along the boundaries of the inset image, and propose a solution in which the boundaries of the inset image are dynamically adapted as a function of the geometry of the scene. This method produces comfortable stereoscopic viewing at a low additional computational cost. The four projectors need only be approximately aligned: a single drawing pass is required, regardless of projector alignment, since the warping is applied as part of the 3-D rendering process.

Citation

Guy Godin, Jean-François Lalonde and Louis Borgeat. Projector-Based Dual-Resolution Stereoscopic Display. IEEE Conference on Virtual Reality, 2004. [PDF], [BibTeX]

Dual-Resolution Stereoscopic Display with Scene-Adaptive Fovea Boundaries

Guy Godin, Jean-François Lalonde and Louis Borgeat

Abstract

We present a multi-projector stereoscopic display which incorporates a high-resolution inset image, or fovea. The system uses four projectors, and the image warping required for on-screen image alignment and foveation is applied as part of the rendering pass. We discuss the problem of ambiguous depth perception between the boundaries of the inset in each eye and the underlying scene, and present a solution where the inset boundaries are dynamically adapted as a function of the scene geometry. An effcient real-time method for boundary adaptation is introduced. It is applied as a post-rendering step, does not require direct geometric computations on the scene, and is therefore practically independent of the size and complexity of the model.

Citation

Guy Godin, Jean-François Lalonde and Louis Borgeat. Dual-Resolution Stereoscopic Display with Scene-Adaptive Fovea Boundaries. Eighth International Immersive Projection Technology Workshop,May 13-14, 2004, Ames, IA, USA. [PDF], [BibTeX]

Augmented-Reality Navigation System for Ground Vehicles (French Only)

Jean-François Lalonde and Jean-Philippe Lajoie-Dorval

Abstract

The goal of this project is to identify important concepts related to the design of augmented-reality navigation systems for ground vehicles. To do so, we first present a state of the current existing technologies, especially the head-up and head-mounted displays. We then present a testing prototype that has been developed to simulate real systems. This allows the study of important factors related to the graphical display of information to the user. The system is tested using various scenarios illustrating its use. Finally, we also developed a real prototype, using a commercial GPS, IMU and webcam. We then apply the algorithms developed previously to this data, and analyze the effect of noise on the end result. Finally, some improvements to the system are proposed. [PDF]