Currently, the robot programming is done either by manual programming or by the "teach-by-showing" method using a teach pendant. Both methods have been found to have several drawbacks. We have been developing a novel method to program a robot, the assembly-plan-from-observation (APO) method. The APO system observes a human performing an assembly task, understands it, and generates a robot program to achieve that same assembly task. Current research topics include how to: choose meaningful snapshots from an image sequence; infer what kind of operation is performed from given snapshots; determine where and from which direction a gripper should grasp an object to achieve a task; and determine a path to transfer an object while avoiding collision with other objects from observation.
Surface inspection is a vision application area that has great industrial significance. Inspection of machined parts, solder joint surfaces, and plastic sheets are just a few examples of industrial tasks too laborious and time consuming for humans. We are developing basic theories for surface inspection as well as complete end-to-end inspection systems. Our theories are ready to be applied to real inspection systems based on rigorous physics and mathematical theories. Current research topic includes: basic surface reflectance mechanisms, temporal-color space representations that relate how illumination variances effect apparent surface colors, and sensing theories for determining the optimal illumination and camera configurations under a given inspection specification. We are also developing complete inspection systems based on the developed theories: photometric sampler for silicone wafer inspection; range-brightness fusion system for solder join inspection; and 4-photometric system for plastic sheet inspection.
Solid modeling is a useful tool for: representing virtual environments for virtual reality systems; representing real environments for robot programming; and modeling real objects for object recognition. Currently, these solid models are constructed by human operators. Often applications provide real objects to be modeled. If we can generate solid models from given real objects, we can reduce the effort and cost of constructing solid models. We are developing techniques to construct solid models from observation of real objects. Research projects include: building a virtual Carnegie Mellon Wean Hall corridor using a movable range finder cart; building human busts from a sequence of images; and building appearance models for vision algorithm compiler.