Martial Hebert became director of the Robotics Institute in November 2014. A native of Chatou, France, Hebert earned his doctorate in computer science at the University of Paris and joined the Carnegie Mellon faculty in 1984. His research has focused on computer vision and perception for autonomous systems, including interpreting both 2-D and 3-D data to build models of environments.

He worked on NavLab, CMU's pioneering program for self-driving vehicles, and more recently led the development of perception capabilities for other robotic platforms.
Hebert has served on the editorial boards of the IEEE Transactions on Robotics and Automation, the IEEE Transactions on Pattern Analysis and Machine Intelligence, and present serves as editor-in-chief of the International Journal of Computer Vision.

He spoke to Link Editor Jason Togyer.

What was your earliest computing experience?

When I was an undergraduate, I was in mathematics, but I took some classes in computer science—I started on punch cards with a Univac, I don’t remember the model—and it was interesting to see how mathematics could be put into practice. So I moved first into applied math, and then into computing.

Did you move from math to computer science because there are more open problems to solve in computer science?

There was certainly a little bit of that. Those also were the days when people were talking about artificial intelligence, and I had the opportunity to be in touch with some of the early labs working on computer vision and robotics—in fact, my advisor was Olivier Faugeras, who became one of the leading names in computer vision research. I was very fortunate to be exposed very early to this sort of thing.

It seems to me that computer vision has moved out of the lab and into people’s everyday lives. What is fueling that move from the theoretical to the practical?

Well, sensors are no longer an expensive proposition. We also now have cameras that have very small form factors and draw very little power. That’s one aspect. The other aspect is computing—by which I mean processing power and storage. And then, of course, there’s all of the progress that has been made in algorithms over the past 30 years. All of those combined make it now possible to have an explosion of commercial applications. And it had to be all three, because even if you had the math but you didn’t have the computing power or the sensors, it wouldn’t get you anywhere.

Is robotics also at a tipping point in terms of commercial applications?

The field is becoming very mature. Thanks in part to the work we’ve done here, industry is doing state-of-the-art research in robotics. That takes away from what we traditionally have done, so we have to adapt to that. It’s a normal process of a field going from purely academic research to maturing into something bigger. As it’s becoming very mature as a field, it requires a lot more skilled workforce, so there’s an opportunity for us in terms of education that is tremendous.

Does that mean growing the graduate programs?

That’s one of the aspects, adding new disciplines. For example, we’re starting a master’s degree in computer vision because computer vision is growing so rapidly. We need to continue to grow our educational programs to respond to the growing need, and we need to continue defining our areas of leadership.

One of the big issues in computer science is privacy. What privacy concerns emerge in computer vision?

We now have technology that not only can track people, but can also recognize emotions. Some of the leading systems can recognize activities and perhaps recognize anomalous behaviors. Once you get into those areas, privacy becomes a very serious concern. We’ve done some work already in those areas—we’ve worked with personal care and nursing robots, for instance, where we had to deal with very deep privacy concerns. Things get even worse when you have systems in someone’s home, and worse and worse when we work with distributed systems and storage.

How do researchers address the concerns?

Well, it’s not just a robotics solution. It’s multi-disciplinary. We concentrate on developing technologies knowing full well that when we go to use them, we’ve got to face those issues, and it takes a lot more than just our people, our work, to do that. We need people from computer science, social science, legal experts, and so forth.

What misconceptions do people have about robotics?

The media’s view of robotics is both good and bad. It’s good because it popularizes what we’re doing, but it raises expectations in directions that are not ones where we should be raising expectations. There are also bad aspects or dangerous aspects of robotics that are portrayed in the media. Perhaps we don’t take that seriously enough. We also still have a lot of work to do to educate people on the fact that robotics is not just humanoid robots. Robotic systems come in many, many different shapes and forms and appearances and have many, many different ways to interact with people. 

What is the next frontier in robotics?

Dependability. One of the things that’s critical for robotic systems to be truly accepted around humans is “safety.” Not safe in the sense of “never fails,” because everything fails at some point, but safe in the sense that you are very confident that you can trust it. When you drive your car, you trust the machine. So what needs to be done to trust a robotics system? They need to be able to detect anomalies in their own behavior, they need to be able to explain and report them. That’s the level of intelligence they need. This translates, by the way, into a large set of very hard technical problems which people have been looking at in different projects, but there’s a need to combine them into a discipline and that’s going to be critical.

There is a very good chance that CMU and Astrobotic will be going to the moon in the near future. What will that mean for the Robotics Institute?

It will say something about how advanced the field is. It says that we are still addressing incredibly challenging problems with the research we do here, but at the same time, we’ve gotten to the point we can do incredibly challenging things, like putting something on the moon.

It’s exciting because in every field, there’s a curve where you have a rapid increase, because you start with nothing, but then it plateaus for a while. We’re not in the plateau. We’re in the very, very rapid increase, and that’s a very nice place to be at this point.

Biomedical imaging systems are developed primarily for diagnostic or scientific measurement rather than for guiding tools and actuators.  Imaging systems suitable for guidance pose unique requirements, including real-time operation, registration between multiple 3D coordinate frames, and device geometries that allow physical access for manipulations and ancillary equipment.  Real-time computer vision algorithms to detect and track targets may be desired, and guidance of human operators additionally requires well-designed visualization and augmented reality interfaces.  Analysis and design of such biomedical image guidance systems will be described, including software, hardware, computer-controlled optics, and human factors research.  This novel work applies and extends recently developed optical and acoustical modalities, including a microsurgical augmented reality system and a unique multi-modal approach to enhancing medical ultrasound of the patient's interior anatomy with simultaneous computer vision of the patient's exterior.


John Galeotti received BS and MS degrees in Computer Engineering from North Carolina State University (2001, 2002), and MS and PhD degrees in Robotics from Carnegie Mellon University (2005, 2007).  He is currently a senior project scientist at Carnegie Mellon University’s Robotics Institute, as well as an adjunct assistant professor with advising privileges at both CMU's Biomedical Engineering department and at the University of Pittsburgh's Bioengineering department.  He directs the Biomedical Image Guidance Laboratory at CMU, and he teaches a graduate course on biomedical image analysis algorithms that was funded by the National Library of Medicine.

Faculty Host: George Stetten

This thesis provides a system that allows users to interact with objects and human subjects in photographs in three dimensions. Current photo-editing tools provide a diverse array of operations in photographs. However, they are limited to the 2D plane of the image. The key contribution of this thesis is to leverage 3D models from public repositories to obtain a three-dimensional representation of objects in photographs for perceptually plausible seamless manipulation of photographed objects. The approach described in this thesis addresses the mismatch between the geometry and appearance of 3D models from photographed instances.

To correct the mismatch between the geometry of the 3D model and the photographed object, the thesis discusses an automatic model alignment technique that uses information from local patches distributed within the object to predict the global transformation and local deformation of the 3D model to the photographed object. We also present a manual geometry adjustment tool that allows users to perform final corrections while imposing smoothness and symmetry constraints. Given the matched geometry, we use the visible pixels to estimate unknown illumination, correct the mismatched appearance, and complete the appearance in hidden areas using symmetries. Our approach provides users with intuitive three-dimensional control over objects to perform manipulations impossible in current photo-editing tools.

Currently, our approach allows users to manipulate inanimate objects in photographs. In this thesis, we propose to provide three-dimensional control over photographed human subjects. Representing human subjects with 3D models is challenging as a single 3D model may not capture all photographed people due to differences in geometric structure and appearance introduced by variation in clothing appearance and style, body shape, and articulation. In the remainder of this thesis, we propose to adapt our automatic model alignment, illumination estimation, and appearance completion approaches to provide three-dimensional manipulation of human subjects in photographs.

Thesis Committee:
Yaser Sheikh (Chair)
Nancy Pollard
Kayvon Fatahalian
Alexei Efros (University of California, Berkeley)
David Forsyth (University of Illinois at Urbana-Champaign)

Copy of Thesis Proposal Document

Dynamic robots with advanced control systems and high-performance mechanical designs are leaving the laboratory and entering the world.  They can operate in rough terrain, where most existing vehicles that use wheels and tracks can not go.  In this talk I will give a status report on the robots we are developing at Boston Dynamics, such as LS3, the DARPA-funded follow-on to BigDog, Cheetah, a fast-running quadruped, and Atlas, an anthropomorphic robot designed to explore real-world tasks.


Marc Raibert is CTO and founder of Boston Dynamics, a company that develops some of the world’s most advanced dynamic robots, such as BigDog, Atlas, Cheetah, SandFlea and the AlphaDog. These robots are inspired by the remarkable ability of animals to move with agility, mobility, speed and grace. Before starting Boston Dynamics, Raibert was Professor of Electrical Engineering and Computer Science at MIT from 1986 to 1995. Before that he was Associate Professor of Computer Science and Robotics Institute at Carnegie Mellon from 1980 to 1986. While at MIT and Carnegie Mellon Raibert founded the Leg Laboratory, a lab that helped establish the scientific basis for highly dynamic legged robots. Raibert earned a PhD from MIT in 1977. He is a member of the National Academy of Engineering.

Faculty Host: Matthew Mason

The Yata Memorial Lecture in Robotics is part of the School of Computer Science Distinguished Lecture Series.  Teruko Yata was a postdoctoral fellow in the Robotics Institute from 2000 until her untimely death in 2002. After graduating from the University of Tsukuba, working under the guidance of Prof. Yuta, she came to the United States. At Carnegie Mellon, she served as a post-doctoral fellow in the Robotics Institute for three years, under Chuck Thorpe. Teruko's accomplishments in the field of ultrasonic sensing were highly regarded and won her the Best Student Paper Award at the International Conference on Robotics and Automation in 1999. It was frequently noted, and we always remember, that "the quality of her work was exceeded only by her kindness and thoughtfulness as a friend." Join us in paying tribute to our extraordinary colleague and friend through this most unique and exciting lecture.


In a lab in the basement of Newell-Simon Hall, robotics doctoral student Matt Tesch grabs what looks like a PlayStation controller and begins quickly pressing buttons and moving the joysticks. A serpentine robot, about two-feet-long and with 16 distinct segments or modules, slithers across the floor like a snake. It stops for a moment and then begins crawling sideways, much like a sidewinder.

After the snakebot crawls on the floor for a bit, showing off all its moves, Tesch grabs a hollow plastic tube, wraps the snakebot around the pole and it rolls upward. After it reaches the top, he takes it off the tube and straightens it out.

He feeds about half of it into the tube, and it inches down slowly.

“We control the motion by sending waves through the joints,” he says.

Tesch is demonstrating a snakebot’s means of locomotion. They can slither backward, crawl through tiny openings and navigate sharp bends, which comes in handy for inspecting pipes. In fact, one recently showed how well it crawls at a nuclear power plant in Austria.

In May, a snakebot slithered through pipes and vents in the never-activated Zwentendorf Nuclear Power Plant, west of Vienna on the south bank of the Danube River. The tests gave researchers an opportunity to access pipes and areas that would be restricted because of high radiation levels in working plants, allowing the robot to go where no robot has gone before.

“Think of how they could have been used at Fukushima,” says Howie Choset, director of the CMU Biorobotics Lab, referring to the Japanese nuclear power plant that melted down in 2011 following an earthquake and tsunami. Radiation levels in much of the plant remain so high that humans can’t enter, and the company that owns the Fukushima Daiichi plant has recently admitted that it doesn’t know where the melted fuel cores are.

But perhaps a snakebot could.

Choset has been developing snakebots for almost two decades. Over the past few years they’ve broken into the public consciousness—and in a big way. Videos of the amazing snakebots from Choset’s lab are going viral, capturing the attention of bloggers and journalists for Popular Science, Bloomberg Businessweek and NPR.

This year’s Austrian tests happened almost through a happy set of coincidences. The Zwentendorf nuclear plant was built in the 1970s. Public opposition to starting the plant meant that it was never commissioned, and instead it’s used for training and educational purposes. Florian Enner, a research associate and software engineer in the Biorobotics Lab, is from Salzburg, Austria, and following a conference in Germany, he asked officials if the snakebot team could do some research at Zwentendorf. To the team’s surprise, the power company that owns Zwentendorf agreed, giving the CMU group access for two entire days.

“In the lab we sort of know what to expect,” Enner says. “There, we didn’t know what we would find.”  As the snakebots crawled through the pipes, they used LED lights and cameras to show the researchers where they were and what they saw. They encountered lost bolts that had lain undetected inside conduits for so long they were rusted to the sides. They found long-forgotten sensors protruding into the pipes.  

The snakebots’ cameras are able to automatically correct the view presented—when they’re upside down, the cameras adjust the video to appear right side up. That feature impressed the power plant’s staff, the researchers say, but more impressive was the robots’ flexibility, which gives them a serious advantage over conventional equipment for inspecting the insides of pipes. Current pipe inspection tools cannot turn corners. When they encounter a sharp bend, workers have to cut a hole in the pipe and push the robot through. It’s awkward, causes damage to pipes and exposes workers to potentially dangerous situations.

A segmented snakebot can easily twist and turn up and into inlets and through tight corners, allowing researchers to be far enough away from dangers such as radiation. (Tesch and Enner have pictures of themselves with the Zwentendorf plant’s steel containment vessel. If the plant were active, they’d never had been able to get so close.)

Inspecting power plants is only one of the many tasks Choset’s snakebots can perform. The robots are effective in search and rescue—snakebots can easily slither between fallen debris in crumbled buildings after a disaster. And snakebots can go places where ladders can’t be used. A rescuer, for instance, can throw the robot at a pole. It grabs the pole, coils around it, and crawls upwards. At a dangerous building, first responders could toss in a snakebot and have it send back images of the scene before humans risk their lives.

“They work well if you want to reach a great distance and thread through tight spaces,” Choset says. “Urban search and rescue use extends your sensory reach to find survivors while protecting first responders.”

Another version of the snakebot—much smaller—is being tested in cardiac heart operations. For many heart procedures, doctors must crack the sternum to access the heart. A simple test could mean a week’s stay in the hospital. But with a snake robot, the same procedures can become minimally invasive. The snakebot can enter a small incision, make a quarter-inch turn, and weave behind the heart, where it can send photos back to physicians. The technology has already been successfully tested on animals. Choset believes that snakebots eventually could allow technicians to perform common procedures that currently require surgeons.

“What’s provocative is that we can have non-surgeons deliver care, and, it may not require a hospital stay,” he says. “Snakebots could decrease the cost of surgery. And robots could create new jobs.”

These robots of the future have gone into the past, too. Prior to the recent Egyptian revolution, Choset took a snakebot to an archeological site near the Red Sea. Ancient Egyptians left their old boats in caves along the coast, but as the centuries passed the caves collapsed, making it impossible for archeologists to reach the sea vessels. One of Choset’s snakebots slipped through the rubble and sent back pictures for the researchers to analyze.  

And are you ready for “snakebots on a plane”? It’s no science fiction spoof: Choset is currently working on a project with Boeing to develop snakebots that could work in manufacturing situations, applying paints and other coatings inside tight spaces, such as tanks. “Snake robots have tons of applications,” he says. “It’s the future of assembly.”

—Meghan Holohan is a Pittsburgh-based freelance writer and a frequent contributor to The Link. Her work also appears at

Mark Palatucci

  • B.S.E., computer science and engineering, University of Pennsylvania, 2000
  • M.S., robotics, Carnegie Mellon University, 2008
  • Ph.D., robotics, Carnegie Mellon University, 2011

Boris Sofman

  • B.S., electrical and computer engineering, Carnegie Mellon University, 2005
  • B.S., computer science, Carnegie Mellon University, 2005
  • M.S., robotics, Carnegie Mellon University, 2007
  • Ph.D., robotics, Carnegie Mellon University, 2010

Hanns Tappeiner

  • Dipl. Ing. Inf., Technische Universität Wien, Austria, 2004
  • M.S., robotics, Carnegie Mellon University, 2008

(Editor’s Note: We’re trying something different with the “Alumni Snapshots” in this issue. We’ve interviewed, together, the three co-founders of San Francisco-based Anki Inc.)

Anki Inc. made its high-profile debut on the world stage June 10 when the company’s first product, Anki Drive, was demonstrated during the keynote at Apple’s Worldwide Developers Conference.

Anki Drive runs on Apple’s iOS and allows users to control toy racecars from their iOS devices. It’s the first game where real, moving objects simultaneously interact with a virtual environment, their physical surroundings and one another.

Netscape co-founder and venture capitalist Marc Andreessen, who serves on the Anki board of directors, calls it “the best robotics startup I have ever seen.”  

Anki’s three co-founders met at Carnegie Mellon in 2005. All of them grew up with an interest in technology—especially robotics.  

“As a kid, I was always interested in making things that could interact with the real world,” says Hanns Tappeiner, who was born in Germany and raised in northern Italy. “One time, I tried to build a machine that could steal candy out of a candy jar. It never really worked! Later, I took up building things in Lego and hooked my creations up to really, really early versions of microcontrollers.”

By the time Tappeiner completed his undergraduate work at Austria’s Technical University of Vienna, he’d been doing robotics “for a very long time,” both as a hobby and as a field of study.

Boris Sofman was born in Russia and immigrated to the United States as a child. His earliest computing experience was programming in Logo, the educational language that allowed users to program either an on-screen turtle or a real-world robot. He came to CMU to earn degrees in both engineering and computer science.

“The idea of making things in the physical world was very exciting to me,” Sofman says. “As an undergrad, I got to participate in a couple of projects at the Field Robotics Center where people were working on autonomous navigation, with robots that could sense and avoid obstacles, and as I was applying to grad schools, I realized the kind of robotics I wanted to study was being done best at CMU.”

Mark Palatucci was just 5 years old when his dad brought an IBM PCjr—the family’s first personal computer—into their Philadelphia home. “I immediately fell in love with it,” he says. “By the time I was 6, I started learning BASIC, and by the time I was 10 years old, my aunt bought me my first robotics kit.” That kit, and others he assembled, are on his desk at Anki 25 years later.

He graduated from Penn, moved to Silicon Valley and started Copera, a company that developed software for handheld PCs and early smartphones. “I also started volunteering on Stanford’s DARPA Grand Challenge team in 2004 to help build the Stanley robot, and met a lot of really incredible people,” including former CMU professor Sebastian Thrun, Palatucci says. “They were all super-smart and they had all come from CMU’s Robotics Institute.” He applied and was accepted into the Ph.D. program.

Besides robotics, the three also shared an interest in consumer products. “Whenever we brainstormed things, it was never about, ‘What can we do in the lab?’” Tappeiner says. “Instead, it was always, ‘What can we do to make this a viable product?’”

The idea that evolved into Anki Drive can be described in four words, according to Sofman. “Real-world Sim City,” he says. “Sim City is an intelligent ecosystem. We wondered how we could make that environment possible in the real world. How could we make a physical object—a car—understand where it was located in its environment, very precisely? How could we make the algorithms efficient enough to do it, and how could we deliver it at a price point that people can afford?”

In laboratory research, Sofman says, it doesn’t matter if a robot needs a $5,000 sensor and “a crazy amount of computation,” but that simply won’t work for a consumer product.

Anki Drive solves the efficiency problem by separating higher-level functions—those that control game play—from less-complex functions. Although the remote cars each have an onboard 50 MHz microcontroller as well as navigational sensors, the artificial intelligence required to play a game is done completely on the user’s iPhone running the Anki Drive app. The cars communicate with the app and one another via Bluetooth LE. Tappeiner compares it to the human body’s separation of autonomous nervous system functions, such as breathing, from conscious, deliberative decisions made by the brain.

Trying to offload all of the decision-making ability to the remote device wouldn’t work, Sofman says. “There’s too much latency, and the bandwidth also wouldn’t support it,” he says.

The separation of functions also ensures the long-term value of the system, Palatucci says. “The mechanical parts onboard the cars only control their basic functionality,” he says. “Over time, we not only can upgrade the software in the app, but we can also upgrade the software used by the microcontrollers in the cars.”

Apple’s iOS 6 ecosystem was the perfect platform for Anki Drive, Palatucci says, because it was one of the first consumer products to support Bluetooth LE, which was designed for low-power consumption and control of multiple devices at the same time.

For anyone who’s enjoyed either conventional remote-control cars or video racing simulations, the prospect of combining the two is appealing. Now imagine scaling up Anki Drive, and using Bluetooth LE and iPads to control larger moving objects—say, robots delivering products in a distribution center, or mass-transit vehicles traveling between stations.

That’s exactly what the Anki co-founders have in mind. While Anki Drive is a finished, sellable product, it’s also a proof-of-concept for a new way of mass-producing semi-autonomous robotic devices.

“From the very beginning, we wanted to make sure Anki was a robotics company, and not a games company or a toy company,” Sofman says. “Entertainment is familiar, it’s fun, and there isn’t a massive number of regulatory barriers, so we thought this was a way to enter the market and re-invent the way people play.

“But the ability to do position-sensing in the real world, and to deploy efficient algorithms that deal with uncertainty, all of those things are in Anki Drive already,” he says. “We want to make the most practical robots that are deliverable today, and then become more and more advanced, tackling larger and larger problems.”

Launching a high-profile startup company has taken its toll on the founders’ personal lives. “I used to have outside interests,” Sofman says, laughing. “It’s gotten a little bit harder lately. I play a lot of tennis, and I’ve started biking. California is a very nice place for that.”

Palatucci learned to fly single-engine private planes while he was at CMU, though like Sofman, he also doesn’t have much time for hobbies these days.

Adds Tappeiner: “Quite frankly, you can’t start a company like this if it’s not also your hobby.”

Key to Anki’s development has been the continuing connection between the company and the School of Computer Science. At this writing, the company employs about four dozen people, one-fourth of whom have ties to the Robotics Institute. “It’s definitely a core part of what we’re doing here, and we’re very thankful for the experience we had at CMU,” says Tappeiner, adding that working with RI research professor Ralph Hollis helped to shape his own ideas.

Palatucci says Tom Mitchell, head of CMU’s Machine Learning Department, was a key influence, while Sofman says RI research professor Tony Stentz and associate professor Drew Bagnell had a big impact on his work.

“We had great advisors and great colleagues,” Sofman says. “What we’ve achieved at Anki on a technical scale was built on things we learned at the Robotics Institute.”

—Jason Togyer (DC’96)

At the rate at which the population of the world is currently growing, it is estimated to reach 10 billion by the year 2083. On the other hand, about 36% of the land suitable for crop production in the world is already being used, leaving only 2.7 billion hectares. The potential of a future shortage has let to rapid acquisitions of vast tracts of foreign land by food-insecure nations to protect the food security of their burgeoning population. We can curb the shortage by intensifying our agricultural techniques and innovating new practices.

Robotics and Hydroponics are two technologies that have shown promising capabilities in the recent years. By automating hydroponic installations we can overcome some of the shortfalls of the system such as the requirement of periodic labor and meticulous monitoring. In addition, the structured environment of greenhouses and systematic arrangement of plants in a hydroponic system make the domain convenient for robots to work in. To compete with the traditional methods, the system has to both produce a high yield with better quality and be inexpensive at the same time.

In this research, we describe our efforts on developing an autonomous robot known as the NFT Bot that automates several tasks that a grower would perform on a hydroponic Nutrient Film Technique (NFT) system. To keep the costs low, the robot was designed to be reconfigurable so that it could be easily deployed on top of any existing NFT infrastructure in a greenhouse. The robot is capable of planting the seedlings in the NFT gullies, manipulating the plants onto to the nursery gullies and harvesting the fully grown plants. The NFT Bot is also equipped with an interchangeable end-effector: one for gripping and manipulating the plants, and another with a Microsoft Kinect for mapping and building colorized 3D models of the plants throughout their grow cycles. Colorized 3D models of the plant structure can be used to estimate different plant properties such as canopy height, leaf area index, ecophysiological responses, plant stress and, etc. In addition, we also use the CMU Sensorweb system on the robot to remotely monitor other plant parameters including temperature, specific conductivity, dielectric constant and water reservoir depth. We present our results from the experiments performed on growing 288 heads of lettuce over two grow cycles using the NFT Bot and the AmHydro 612HL NFT system.


Abhinav Valada is a Masters student at the Robotics Institute, advised by George Kantor and Paul Scerri. He is also an Engineer at the National Robotics Engineering Center and a co-founder of the CMU spin-off Platypus, LLC. Prior to working at NREC, he was a Systems/Software Engineer at the Field Robotics Center. He received his Bachelors in Electronics and Instrumentation Engineering from VIT University.

This seminar will serve as the masters talk for the speaker.

Many computer vision problems, such as object classification, motion estimation or shape registration rely on solving the correspondence problem. Existing algorithms to solve the correspondence problems are usually NP-hard, difficult to approximate and lack mechanism for feature selection. This proposal addresses the correspondence problem in computer vision, and proposes two new correspondence problems and three algorithms to solve spatial, temporal and spatio-temporal correspondence problems. The main contributions of the thesis are:

(1) Factorial graph matching (FGM). FGM extends existing work on graph matching (GM) by finding an exact factorization of the affinity matrix. Four are the benefits that follow from this factorization: (a) There is no need to compute the costly (in space and time) pairwise affinity matrix; (b) It provides a unified framework that reveals commonalities and differences between GM methods. Moreover, the factorization provides a clean connection with other matching algorithms such as iterative closest point; (c) The factorization allows the use of a path-following optimization algorithm, that leads to improved optimization strategies and matching performance; (d) Given the factorization, it becomes straight-forward to incorporate geometric transformations (rigid and non-rigid) to the GM problem.

(2) Canonical time warping (CTW). CTW is a technique to temporally align multiple multi-dimensional and multi-modal time series. CTW extends DTW by incorporating a feature weighting layer to adapt different modalities (e.g., video and motion capture data), allowing a more flexible warping as combination of monotonic functions, and has linear complexity (unlike DTW that has quadratic). We applied CTW to align human motion captured with different sensors (e.g., audio, video, accelerometers).

 (3) Spatio-temporal matching (STM). STM simultaneously finds the temporal and spatial correspondence between trajectories of multidimensional multi-modal time series. STM is used to solve for spatial and temporal correspondence between 2D videos and 3D data (e.g., motion capture data or Kinect).

Thesis Committee:
Fernando De la Torre (Chair)
Martial Hebert
Jeff Schneider
Anand Rangarajan (University of Florida)

Copy of Proposal Document

Can machines discover scientific laws automatically? Despite the prevalence of computing power, the process of finding natural laws and their corresponding equations has resisted automation. This talk will outline a series of recent research projects, starting with self-reflecting robotic systems, and ending with machines that can formulate hypotheses, design experiments, and interpret the results, to discover new scientific laws. We will see examples from psychology to cosmology, from classical physics to modern physics, from big science to small science.


Hod Lipson is an Associate Professor of Mechanical & Aerospace Engineering and Computing & Information Science at Cornell University in Ithaca, NY. He directs the Creative Machines Lab, which focuses on novel ways for automatic design, fabrication and adaptation of virtual and physical machines. He has led work in areas such as evolutionary robotics, multi-material functional rapid prototyping, machine self-replication and programmable self-assembly. Lipson received his Ph.D. from the Technion - Israel Institute of Technology in 1998, and continued to a postdoc at Brandeis University and MIT. His research focuses primarily on biologically-inspired approaches, as they bring new ideas to engineering and new engineering insights into biology.

Faculty Host: Ralph Hollis

Emerging robots have the potential to improve healthcare delivery, from enabling surgical procedures that are beyond current clinical capabilities to autonomously assisting people with daily tasks in their homes. In this talk, we will discuss new algorithms to enable medical and assistive robots to safely and semi-autonomously operate inside people’s bodies or homes. These algorithms must compensate for uncertainty due to variability in humans and the environment, consider deformations of soft tissues, guarantee safety, and integrate human expertise into the motion planning process.

First, we will discuss how our new algorithms apply to two recently created medical devices for neurosurgery and cardiothoracic surgery: steerable needles and tentacle-like robots. These new devices can maneuver around anatomical obstacles to perform procedures at clinical sites inaccessible to traditional straight instruments. To ensure patient safety, our algorithms explicitly consider uncertainty in motion and sensing to maximize the probability of avoiding obstacles and successfully accomplishing the task. We compute motion policies by integrating physics-based biomechanical simulations, optimal control, parallel computation, and sampling-based motion planners. Second, we will discuss how our new algorithms apply to autonomous robotic assistance for tasks of daily living in the home. We will present demonstration-guided motion planning, an approach in which the robot first learns time-dependent features of an assistive task from human-conducted demonstrations and then autonomously plans motions to accomplish the learned task in new environments with never-before-seen obstacles.

Research group web site


Dr. Ron Alterovitz is an Assistant Professor in Computer Science at the University of North Carolina at Chapel Hill. He leads the Computational Robotics Research Group which investigates new algorithms to enable robots to safely and autonomously complete novel tasks in clinical and home environments. Prior to joining UNC-Chapel Hill in 2009, Dr. Alterovitz earned his B.S. with Honors from Caltech, completed his Ph.D. at the University of California, Berkeley, and conducted postdoctoral research at the UCSF Comprehensive Cancer Center and the Robotics and AI group at LAAS-CNRS (National Center for Scientific Research) in Toulouse, France. Dr. Alterovitz has co-authored a book on Motion Planning in Medicine, was co-awarded a patent for a medical device, and has received multiple best paper finalist awards at IEEE robotics conferences. He is the recipient of an NIH Ruth L. Kirschstein National Research Service Award, the UNC Computer Science Department’s Excellence in Teaching Award, and an NSF CAREER award.

Faculty Host: Howie Choset


Subscribe to Robotics