Paul E. Rybski Back to Home Page

Research Interests
I am interested in the scientific challenges involved with solving complex tasks with heterogeneous distributed autonomous systems that can be comprised of both stationary sensors as well as mobile robots. Additionally, I am interested in the research questions involved with the interactions between humans and these autonomous systems, particularly when the human and the robot must actively work together as partners in a task. In order to operate effectively in the real world, such systems must observe, model, and infer the internal state of the world (including other robotic and human agents) around them. To succeed with such inference tasks, I believe that autonomous systems must take an active role in intelligently exploring their environment to explicitly search for the information that will further their goals. One challenge that must be addressed in this research is to define methods for robust sensor interpretation and effective state estimation. Towards this end, I have developed algorithms and techniques for robust spatial reasoning for localization and SLAM, as well as local sensory-based control and navigation systems for small and difficult to model autonomous robotic systems. A second challenge that must be addressed is to define how the relative strengths of each team member will complement each other such that the team functions better as a coordinated group. This involves identifying how each member of the team can operate within its partition of the task, as well as determining the kind and amount of communication necessary to achieve good collaboration. Towards this end, I have developed algorithms for multi-robot shared world models, developed algorithms for cooperation that involve communication through the environment, analyzed the effect of sharing limited bandwidth communication channels, and analyzed the effect of communication duration for recruiting in search and retrieval. Finally, effective use of communication and state estimation is required to interact robustly with people. Some of my most recent work has included mechanisms for autonomous human activity recognition and allowing a human to teach a robot how to do a task through a combination of physical demonstration of the task and spoken language dialog with the robot. For more info see my publications list.
Research at Carnegie Mellon University
I joined the CORAL in the Robotics Institute at Carnegie Mellon University as a postdoctoral fellow in August of 2003. My postdoctoral advisor was Prof. Manuela Veloso. In July of 2006, I joined the faculty of The Robotics Institute as a System Scientist. I continue to be affiliated with Prof. Veloso and the CORAL research group.
Tartanracing In 2007, I joined CMU's entry in the DARPA Urban Challenge as the head of the perception effort. We are designing and building a robotic car that will be able to navigate through city streets, understand and follow the rules of the road, and handle traffic situations. To see more, visit the Tartanracing home page.	The Tartanracing logo.
CMAssist In 2006, a new RoboCup league called RoboCup@Home was formed. This league is a departure from traditional RoboCup leagues which focused on research in a Soccer domain. Instead, RoboCup@Home focuses on socially-relevant human/robot interaction (HRI) in a home environment. To particpate in this league, I formed the CMAssist RoboCup@Home team. To see more, visit the CMAssist page.	Erwin and Carmela, the CMAssist robots.
CAMEO (CALO-PA) In order to effectively interact with humans in a natural fashion, an intelligent agent must have a robust physical awareness system with which it can sense humans as they perform tasks in the real world. The Camera Assisted Meeting Event Observer (CAMEO), is an omnidirectional sensory system designed to provide an intelligent computer agent with physical awareness of the real world. As a stationary camera, CAMEO has the advantages of only needing to model and track objects around it rather than having to estimate its own motion as well. However, very few assumptions can be made about the exact structure of the environment, or CAMEO's absolute position within it. I have developed an action recognition algorithm which uses the tracked relative motions of human faces as input. This algorithm makes a number of assumptions about the basic models of human behavior, but has the advantage of being able to operate within many different environments. This technique is now being used to learn specific objects in the environment based on people's activities. Through this functional object recognition algorithm, we are able to model objects associated with the tracked activities, e.g. chairs by identifying the objects where people sit. The CAMEO project is part of a much larger effort called CALO (Cognitive Agent that Learns and Organizes) To see more, visit the CAMEO page.	The CAMEO sensor.
Segway Soccer The SegwayRMP project investigates the coordination of dynamically formed, mixed human-robot teams within the realm of a team task that requires real-time decision making and response. We are working towards the realization of Segway Soccer, which is a game of soccer between two teams consisting of Segway riding humans and Segway RMP-based robots. The SegwayRMP's sensors consist of a servo-mounted color camera and has a pneumatic kicker with an actuated ball manipulation device. I have developed an Extended Kalman Filter SLAM implementation for the Segway which allows it to stay within the boundaries of the field as well was helping to develop the initial software infrastructure for the Segway's perceptual systems. I am interested in extending my work in human activity recognition into this domain to allow humans and robots to cooperatively play soccer. To see more, visit the Segway page.	Segway soccer.
AIBO Soccer We are investigating the complexities of dynamic real-time adversarial environments with an AIBO RoboCup soccer team. In the RoboCup legged league, teams of four AIBOs compete against one another in two ten-minute halves. Because the AIBOs are quadrupedal robots, their motions are extremely difficult to model correctly, particularly when they are being jostled by other players. The low viewing angle and narrow field of view of their cameras make it difficult to get an accurate world view of all objects around them. I am primarily interested in addressing the problems of using communication to create an accurate shared world model in the face of very noisy sensors and actuators. An accurate shared world model is extremely important for effective teamwork as the robots can model the effects of their teammate's actions and plan accordingly. As part of my work with the Robot Soccer group in Fall 2003, I assisted with a course being taught with the Sony AIBO. The course materials from 2003 can be downloaded here. The 2004 materials will be available early in January 2005.	CMPack AIBOs.
Graduate Research at the University of Minnesota
I did my graduate work, both MS and Ph.D, in the Department of Computer Science and Engineering at the University of Minnesota. I worked in the AIRVL - AI, Robotics, and Vision Laboratory for my advisor, Dr. Maria Gini.
Building Topological Maps with Minimalistic Sensor Models I defended my Ph.D dissertation on May 23, 2003. My thesis advisor was Dr. Maria Gini. My committee members were Dr. Nikolaos Papanikolopoulos, Dr. Stergios Roumeliotis, Dr. Richard Voyles, and Dr. Dan Kersten. My dissertation addresses the problem of simultaneous localization and mapping for miniature robots that have extremely poor odometry and sensing capabilities. Existing robotic mapping algorithms generally assume that the robots have good odometric estimates and have sensors that can return the range or bearing to landmarks in the environment. This work focuses on solutions to this problem for robots where the above assumptions do not hold. A novel method is presented for a sensor poor mobile robot to create a topological estimate of its path through an environment by using the notion of a virtual sensor that equates ``place signatures'' with physical locations in space. The method is applicable in the presence of extremely poor odometry and does not require sensors that return spatial (range or bearing) information about the environment. Without sensor updates, the robot's path estimate will degrade due to the odometric errors in its position estimates. When the robot re-visits a location, the geometry of the map can be constrained such that it corrects for the odometric error and better matches the true path. Several maximum likelihood estimators are derived using this virtual sensor methodology. The first estimator uses a physics-inspired mass and spring model to represent the uncertainties in the robot's position and motion. Errors are corrected by relaxing the spring model through numerical simulation to the state of least potential energy. The second method finds the maximum likelihood solution by linearizing a Chi-squared error function. This method has the advantage of explicitly dealing with dependencies between the robot's linear and rotational errors. Finally, the third method employs the iterated form of the Extended Kalman Filter. This method has the advantage of providing a real-time update of the robot's position where the others process all the data at once. Finally, a method is presented for dealing with multiple locations that cannot be disambiguated because their signatures appear to be identical. In order to decide which sensor readings are associated with what positions in space, the robot's sensor readings and motion history are used to calculate a discrete probability distribution over all possible robot positions. Feel free to download and read my PhD Dissertation (10M pdf).	Appearance-based mapping.
Distributed Robotics In 1999, I became involved with a project to develop a distributed team of robots for surveillance and reconnaissance applications. This robotic team consists of two kinds of robots, the scouts and the rangers. The scouts are tiny cylindrical robots 11cm in length and 4cm wide. They can roll along on the floor with two wheels and can hop over obstacles using a leaf-spring foot. They carry sensor payloads (usually a camera) to sense their environments with. The scout is a completely original robot designed (and patented) by researchers at the University of Minnesota, MTS Systems Corp, Architecture Technology Corp. and Honeywell. Scouts are carried and deployed by the larger rangers, the second kind of robot in the team. Rangers are commercially-available off-road platforms from iRobot that we have modified and added our own accessories to. Each ranger carries a custom scout launcher (also designed and built at the University of Minnesota) which it uses to deploy scouts into an environment. I developed an automated resource scheduling system that allowed multiple control processes to access the hardware necessary to control the miniature Scout robots. This system keeps track of current resource allocations and limits and schedules access to resources that are overextended. This resource allocation system was used to manage the robots in the automated placement of a sensor network. We analyzed and modeled the performance of this system from a rigorous experimental standpoint in an attempt to try to answer the question of how many robots would be needed to do this task given a simple description of the world. Finally, I developed the control electronics for a ballistic Scout deployment launcher and a microphone sensor for the Scout robots. For more info, see the Distributed Robotics home page.	Here is a scout shown next to a quarter for scale. Here is a ranger carrying a Scout Launcher.
Minnesota Distributed Autonomous Robot Team (MinDART) From 1997 to the present, our research group has been interested in simple multi-agent foraging techniques. We constructed this group of LEGO-based robots in order to examine the kinds of problems that the environment, intelligence of the robot and distribution of target objects in the environment affect the performance of the group. This team has been used to analyze the kinds of engineering decisions that typically go into creating a distributed robotic system to solve a particular task. We ran more than 100 individual experiments that tested the performance of the robotic team under different environmental conditions, communication strategies, and control strategies. Some of the results from these experiments were that for this kind of simple task, the effects of randomness in the behaviors of the robots typically outweigh any attempts to boost performance due to communication and control strategy. Instead of a statistically-significant global performance boost, we observed that additional knowledge in the system can have a significant effect on the variance of the system. These robots are programmed with the MIT Handyboard. Earlier versions of the robots used simple microswitches, IR detectors and photoresistors to sense their environment, while the current versions make use of the CMUCam and can signal each other via the use of light beacons. For more info, larger pictures, and movies, see the mindart home page.	A closeup photo of a single robot. The group of five robots with their targets. A closeup of the Infrared-transmitting target.
Pioneer Robots Our group owns a group of five ActivMedia Pioneer 1 robots as well as several Pioneer 2ATs. All of the robots Pioneer 1s are equipped with grippers and two of the Pioneer 1s can make use of the Newton Labs Cognachrome Vision System. We have upgraded our vision system to 1 Meg of RAM, which allows us to track more than 3 colors (although only three can be tracked simultaneously). One of our Pioneer 1s has a custom pan/tilt unit for its camera that is run by the gripper's servo controller. We have several SICK laser range finders and can mount these on any of the robots. One of our Pioneer 2s is equipped with a custom panoramic vision system (an omnicamera). In 1999, I did a project with these robots in human/robot interaction and task training. This project focused on exploring the how a human being can interact with a simple mobile robot and teach it how to accomplish a simple mobile manipulation task. A Human Robot Interface (HRI) was developed in order to allow a human to show a robot how to solve a particular task in real time. The robot used a set of Hidden Markov Models (HMMs) to visually classify the actions carried out by the human ``teacher''. Because this task was performed on a mobile robot, it was able to follow the human teacher around the environment and observe how the teacher interacted with objects in that environment. By coupling observation and mobility, the robotic was able to implicitly couple actions with specific locations in the environment. For more info, see my gesture-based programming page.	A Pioneer 1 with SICK laser, USB camera and Pentium III laptop running Linux. Four of our Pioneer 1s equipped the same way as above. A Pioneer 2 AT with a custom omnicamera.
Trailer Backer During the 1996-1997 school year, I built a robotic cart and trailer setup to run a neural network learning system designed by Dean Hougen. One of the most difficult things to do when driving a car with a trailer attached to it is to back it up a driveway or down a boat launch. Therefore, we thought we'd try to implement a learning system capable of doing this very task. The trailer has a tracking turret which keeps track of where a light bulb (the target) is. After about 8 or 9 tries, the robot is usually able to figure out how to hit the target under a few different circumstances. After about 50-60 tries, the robot can back up to the light from any configuration. For more info see the Trailer Backer page.	The 2nd generation Trailer Backer. The 3rd generation Trailer Backer. 3rd generation with 2 trailers.
Loon In the summer of 1996, I worked on a team of people which included Brian Schmalz, Dirk Edmonds, and Margaret Hsieh, to build a multi-robot entry for one of the robotics competitions at the AAAI conference held in Portland, Oregon. Loon was a system which consisted of two robots that had the same configuration of sensors and actuators. However, one was about 1/2 the size of the other one and could ride on the larger one's back. Communication was accomplished through a serial-line cable and the larger robot could deploy the smaller one when needed. For more info see the Loon page.	This is the Loon multi-robot system.
Surfbot In winter 1996, I developed an implementation of an Artificial Potential Field algorithm that ran on a small LEGO robot as my final project for CSCI 5512, Artificial Intelligence II. For more info go to my Surfbot page.	Surfbot -- A robot which navigates by potential fields.
Walleye In the 1995-96 school year, I assisted with the Walleye autonomous vision project. One undergrad student developed the vision code itself and I worked on some control/search algorithms. For more info see the Walleye page.	This is Walleye, built by John Fischer and other students in the AIRVL.
Autonomous LEGO 'bots The first robots that I worked with consisted primarily of LEGO parts with integrated sensor electronics and hobby servos. For control, I used two different 68HC11-based microcontrollers: the Handyboard and the Miniboard. For more info, see the Mini Robots home page.	6-leg walker is a simple LEGO robot designed to demonstrate a very simple dual-tripod gait.

Last updated April 03, 2007