The capacity of aerial robots to autonomously explore, inspect and map their environment is key to many applications. This talk will overview and discuss a set of new -and in their majority open sourced and experimentally verified- sampling-based strategies that break new ground on how a robot can efficiently inspect a structure for which a prior model exists, how to explore unknown environments, and how to actively combine the planning and perception loops to achieve autonomous exploration with maintained levels of 3D mapping fidelity. In particular, we will detail recent developments in the field of active perception and belief-space planning for autonomous exploration. Finally, an overview of further research activities on aerial robotics, including solar-powered unmanned aerial vehicles and aerial manipulators will be provided.

Kostas Alexis obtained his Ph.D. in the field of aerial robotics control and collaboration from the University of Patras, Greece in 2011. His Ph.D. research was supported by the Greek national-European Commission Excellence scholarship. After successfully defending his Ph.D. thesis, he was a awarded a Swiss Government fellowship and moved to Switzerland and ETH Zurich. From 2011 to June 2015 he held the position of senior researcher at the Autonomous Systems Lab, ETH Zurich, leading the lab efforts in the fields of control and path planning for advanced navigational and operational autonomy. His research interests lie in the fields of control, navigation, optimization and path-planning focusing on aerial robotic systems with multiple and hybrid configurations.

He is the author or co-author of more than 50 scientific publications and has received several best paper awards and distinctions, including the IET Control Theory & Applications Premium Award 2014. Furthermore, together with his collaborators, they have achieved world records in the field of solar-powered flight endurance. Kostas Alexis has participated in and organized several large-scale multi-million dollar research projects with broad international involvement and collaboration. In July 2015, Kostas moved to the University of Nevada, Reno with the goal to dedicate his efforts towards establishing true autonomy for aerial and other kinds of robotics.

Faculty Host: Sebastian Scherer

This work describes methods for advancing the state of the art in mobile robot navigation and physical Human-Robot Interaction (pHRI). An enabling technology in this effort is the ballbot, a person-sized mobile robot that balances on a ball. This underactuated robot presents unique challenges in planning, navigation, and control; however, it also has significant advantages over conventional mobile robots. The ballbot is omnidirectional and physically compliant. Moving requires the ballbot to lean, but this also gives it the ability to achieve both soft, compliant physical interaction and apply large forces.

The work presented in this thesis demonstrates the ability to navigate cluttered environments with the ballbot. Formulating the system as differentially flat enables fast, analytic trajectory planning. These trajectories are used to plan in the space of static and dynamic obstacles. Leveraging the ballbot’s navigational capabilities, this thesis also presents a method of physically leading people by the hand. A hu- man subject trial was conducted to assess the feasibility, safety, and comfort of this method. This study was successful, with the ballbot leading participants to multiple goals utilizing an amount of force that users found comfortable.

Another area of pHRI explored in this thesis is assisting people in transition from a seated position to standing. Another user study was conducted to discover how humans help each other out of chairs and how much force they apply. These data were used to design an impedance controller for the ballbot, and this controller was tested and found to deliver equivalent forces to those generated by people.

Lastly, this work explores capabilities that could enable the ballbot to navigate through dense crowds of people. A method for detecting collision and estimating external forces was explored. This method was tested and used to modify a costmap. Iteratively updating this costmap and using it to plan trajectories enabled the robot to discover obstacles through collision. Because the ballbot is inherently compliant, these collisions resulted in safe interactions with small forces.

Thesis Committee:
Ralph Hollis (Chair)
George Kantor
Jodi Forlizzi
Bill Smart (Oregon State University)

Copy of Thesis Draft Document

Robots today have the capability to collect terabytes of data about their environment and travel kilometers in a single day, yet they are still constrained by one fundamental resource: time. Time limits the number of samples a robot can collect, sites it can analyze, and data it can return for review, so it is imperative the rover makes intelligent actions when it comes to choosing when, where, and what to sample, a process known as adaptive sampling.

In this work we propose an approach to modeling and adaptive sampling. We consider a scenario in which a rover collects a set of a set of noisy, quickly-gathered measurements and analyzes it to select targets for longer follow-up operations. These two types of measurements might come from different devices, such as in an agricultural scenario where images of leaves are quickly analyzed to decide whether or not to follow up with a drawn-out yet informative spectra, or they might come from the same device, such as evaluating a quick sensor measurement and increasing the integration time of a follow-up measurement as desired. Our goal is to vastly decrease the amount of time needed to understand a scene by minimizing the number of samples we have to collect and the time spent collecting each sample.

We propose an approach in which we make estimates of models of materials within a scene, then perform follow-up measurements that validate these models and explore areas in which they do not generalize well. We have no prior knowledge about the number of distinct materials within scenes or how similar they are to materials we’ve observed before. In addition to choosing where to sample, we also propose methods for deciding how long to collect measurements or how many measurements to take. Using the models we have estimated in our first step, we decide how intensively we analyze further samples from those models and whether or not it is profitable to continue sampling at that location. We test this work on three main scenarios at vastly different scales, although the approach generalizes well to a number of other domains. First, we consider a rover analyzing low-resolution orbital data and selecting a path that maximizes the diversity of sampling locations. Second, we consider the scenario in which a rover collects an image and chooses objects within that image for further sampling. Finally, we consider the scenario in which a sensor, such as a spectrometer, collects quick and noisy measurements of a small patch of material, the selects a subset of sampled points for further analysis.

Thesis Committee:
David Wettergreen (Chair)
Jeff Schneider
George Kantor
David R. Thompson (Jet Propulsion Laboratory)

Copy of Thesis Document

Robots can act as a force multiplier for people, whether a robot assisting an astronaut with a repair on the International Space station, a UAV taking flight over our cities, or an autonomous vehicle driving through our streets. To achieve complex tasks, it is essential for robots to move beyond merely interacting with people and toward collaboration, so that one person can easily and flexibly work with many autonomous robots. The aim of my research program is to create autonomous robots that collaborate with people to meet their needs by learning decision-theoretic models for communication, action, and perception.

Communication for collaboration requires models of language that map between sentences and aspects of the external world. My work enables a robot to learn compositional models for word meanings that allow a robot to explicitly reason and communicate about its own uncertainty, increasing the speed and accuracy of human-robot communication. Action for collaboration requires models that match how people think and talk, because people communicate about all aspects of a robot's behavior, from low-level motion preferences (e.g., "Please fly up a few feet") to high-level requests (e.g.,"Please inspect the building"). I am creating new methods for learning how to plan in very large, uncertain state-action spaces by using hierarchical abstraction. Perception for collaboration requires the robot to detect, localize, and manipulate the objects in its environment that are most important to its human collaborator. I am creating new methods for autonomously acquiring perceptual models in situ so the robot can perceive the objects most relevant to the human's goals. My unified decision-theoretic framework supports data-driven training and robust, feedback-driven human-robot collaboration.

Stefanie Tellex is an Assistant Professor of Computer Science and Assistant Professor of Engineering at Brown University. Her group, the Humans To Robots Lab, creates robots that seamlessly collaborate with people to meet their needs using language, gesture, and probabilistic inference, aiming to empower every person with a collaborative robot. She completed her Ph.D. at the MIT Media Lab in 2010, where she developed models for the meanings of spatial prepositions and motion verbs. Her postdoctoral work at MIT CSAIL focused on creating robots that understand natural language.

She has published at SIGIR, HRI, RSS, AAAI, IROS, ICAPs and ICMI, winning Best Student Paper at SIGIR and ICMI, Best Paper at RSS, and an award from the CCC Blue Sky Ideas Initiative. Her awards include being named one of IEEE Spectrum's AI's 10 to Watch in 2013, the Richard B. Salomon Faculty Research Award at Brown University, a DARPA Young Faculty Award in 2015, and a 2016 Sloan Research Fellowship. Her work has been featured in the press on National Public Radio, MIT Technology Review, Wired UK and the Smithsonian. She was named one of Wired UK's Women Who Changed Science In 2015 and listed as one of MIT Technology Review's Ten Breakthrough Technologies in 2016.

Faculty Host: Manuela Veloso

A recurrent and elementary machine perception task is to localize objects of interest in the physical world, be it objects on a warehouse shelf or cars on a road. In many real-world examples, this task entails localizing specific object instances with known 3D models. For example, a warehouse robot equipped with a depth sensor is required to recognize and localize objects in a shelf with known inventory, while a low-cost industrial robot might need to localize parts on an assembly line.

Most modern-day methods for the 3D multi-object localization task employ scene-to-model feature matching or regression/classification by learners trained on synthetic or real scenes. While these methods are typically fast in producing a result, they are often brittle, sensitive to occlusions, and depend on the right choice of features and/or training data. This thesis introduces and advocates a deliberative approach, where the multi-object localization task is framed as an optimization over the space of hypothesized scenes. We conjecture that deliberative reasoning--such as understanding inter-object occlusions--is essential to robust perception, and that the role of discriminative algorithms should mainly be to guide this process.

As part of this thesis work so far, we have developed two methods towards this objective: PErception via SeaRCH (PERCH) and Discriminatively-guided Deliberative Perception (D2P). PERCH exploits structure in the optimization over hypothesized scenes to cast it as a tree search over individual object poses, thereby overcoming the computational intractability of joint optimization. D2P extends PERCH by allowing modern statistical learners such as deep neural networks to guide the global search. This is made possible by Multi-Heuristic A* (MHA*) and its extensions, graph search algorithms which we developed for handling multiple, possibly "inadmissible" heuristics. These algorithms allow us to leverage arbitrary learning-based algorithms as heuristics to accelerate search, without compromising on solution quality.

Our experiments with D2P indicate that we can leverage the complementary strengths of fast learning-based methods and deliberative classical search to handle both "hard" (severely occluded) and "easy" portions of a scene by automatically sliding the amount of deliberation required. For easy scenes, the algorithm mostly relies on learning-based methods to save computation, while for harder scenes, it injects more deliberation to gain robustness at the expense of computation time. In addition, to demonstrate the applicability of D2P to real-world perception tasks, we have integrated our method with the Human-Assisted Robotic Picker (HARP)--the system that represented CMU at the 2016 Amazon Picking Challenge. For the remaining portion of this thesis work, we first propose to study whether D2P can achieve real-time performance, independently of the complexity of the scene. Further, our existing approach assumes that there is no extraneous clutter, and that the objects have only 3 degrees of freedom. In the remainder of this thesis, we aim to relax these assumptions to permit broader applicability of Deliberative Perception.

Thesis Committee:
Maxim Likhachev (Chair)
Martial Hebert
Siddhartha S. Srinivasa
Manuela Veloso
Dieter Fox (University of Washington)

Copy of Proposal Document

The advent of robotic systems to medicine has revolutionized the practice of surgery. Most recently, several novel robotic surgical systems have been developed are entering the operative theater. This lecture will describe the current state-of-the-art in the robotic surgery. We will also describe some of the newer systems that are currently in use. Finally, the future of robotic surgery will be described in the context of clinical development and ease of use in the operating theaters of the future.

Umamaheswar Duvvuri, MD, PhD, is a graduate of the University of Pennsylvania obtaining his Medical Degree in 2000 and his PhD in Biophysics in 2002. He completed an internship in General Surgery in 2003 and residency training in Otolaryngology in 2007 at the University of Pittsburgh Medical Center. He completed fellowship training in Head and Neck Surgery in 2008 at the University of Texas MD Anderson Cancer Center. He joined the University of Pittsburgh in August 2008 as an Assistant Professor in the Department of Otolaryngology, Head and Neck Surgery Division and is also a staff physician in the VA Pittsburgh healthcare System. He serves as the Director of Robotic Surgery, Division of Head and Neck Surgery, at the University Of Pittsburgh School Of Medicine and is the current Director of the Center for Advanced Robotics Training (CART) at the University of Pittsburgh Medical Center. He directs the Cart Training Courses which provide technical and circumstantial resources to initiate and optimize robotic surgery programs. He has authored numerous research publications and book chapters and is an invited guest lecturer/speaker on the subject of robotic surgery both nationally and internationally. A Fulbright scholar, his research interests include minimally invasive endoscopic and robotic surgery of the head and neck, tumors of the thyroid and parathyroid glands and molecular oncology of head and neck cancer. He is a leader in his field and has proctored Transoral Robotic Surgery cases at numerous medical educational facilities throughout the United States and Europe. He directs a federally funded laboratory that studies the biology of head and neck cancer. He holds funding from the National Institute of Health, Department of Veterans Affairs and the “V” foundation.

Faculty Host: Howie Choset
Appointments: Peggy Martin (pm1e@andrew.cmu.edu)

As we work to move robots out of factories and into human environments, we must empower robots to interact freely in unstructured, cluttered spaces. Humans do this easily, using diverse, whole-arm, nonprehensile actions such as pushing or pulling in everyday tasks. These interaction strategies make difficult tasks easier and impossible tasks possible.

In this thesis, we aim to enable robots with similar capabilities. In particular, we formulate methods for planning robust open-loop trajectories that solve the rearrangement planning problem using nonprehensile interactions. In these problems, a robot must plan in a cluttered environment, reasoning about moving multiple objects in order to achieve a goal.

The problem is difficult because we must plan in continuous, high-dimensional state and action spaces. Additionally, during planning we must respect the physical constraints induced by the nonprehensile interaction between the robot and the objects in the scene.

Our key insight is that by embedding physics models directly into our planners we can naturally produce solutions that use nonprehensile interactions such as pushing. This also allows us to easily generate plans that exhibit full arm manipulation and simultaneous object interaction without the need for programmer defined high-level primitives that specifically encode this interaction. We show that by generating these diverse actions, we are able to find solutions for motion planning problems in highly cluttered, unstructured environments.

In the first part of this thesis we formulate the rearrangement planning problem as a classical motion planning problem. We show that we can embed physics simulators into randomized planners. We propose methods for reducing the search space and speeding planning time in order to make the planners useful in real-world scenarios.

The second part of the thesis tackles the imperfect and imprecise worlds that reflect the true reality for robots working in human environments. We pose the rearrangement planning under uncertainty problem as an instance of conformant probabilistic planning and offer methods for solving the problem. We demonstrate the effectiveness of our algorithms on two platforms: the home care robot HERB and the NASA rover K-Rex.

We demonstrate expanded autonomous capability on HERB, allowing him to work better in high clutter, completing previously infeasible tasks and speeding feasible task execution. In addition, we show these planners increase autonomy for the NASA rover K-Rex by allowing the rover to actively interact with the environment.

Thesis Committee:
Siddhartha S. Srinivasa (Chair)
Matthew T. Mason
Maxim Likhachev
David Hsu (National University of Singapore)
Terrence W. Fong (NASA Ames Research Center)

Copy of Thesis Document

Data driven approaches to modeling time-series are important in a variety of applications from market prediction in economics to the simulation of robotic systems. However, traditional supervised machine learning techniques designed for i.i.d. data often perform poorly on these sequential problems. This thesis proposes that time series and sequential prediction, whether for forecasting, filtering, or reinforcement learning, can be effectively achieved by directly training recurrent prediction procedures rather then building generative probabilistic models.

To this end, we introduce a new training algorithm for learned time-series models, Data as Demonstrator (DaD), that theoretically and empirically improves multi-step prediction performance on model classes such as recurrent neural networks, kernel regressors, and random forests. Additionally, experimental results indicate that DaD can accelerate model-based reinforcement learning. We next show that latent-state time-series models, where a sufficient state parametrization may be unknown, can be learned effectively in a supervised way. Our approach,  Predictive State Inference Machines (PSIMs), directly optimizes – through a DaD-style training procedure – the inference performance without local optima by identifying the recurrent hidden state as a predictive belief over statistics of future observations. Fundamental to our learning framework is that the prediction of observable quantities is a lingua franca for building AI systems. We propose three extensions that leverage this general idea and adapt it to a variety of problems. The first aims to improve the training time and performance of more sophisticated recurrent neural networks. The second extends the PSIM framework to controlled dynamical systems. The third looks to train recurrent architectures for reinforcement learning problems. 

Thesis Committee:
J. Andrew Bagnell (Co-chair)
Martial Hebert (Co-chair)
Jeff Schneider
Byron Boots (Georgia Institute of Technology)

Copy of Proposal Document

It is a paradox that often the more severe a person's motor impairment, the more challenging it is for them to operate the very assistive machines which might enhance their quality of life. A primary aim of my lab is to address this confound by incorporating robotics autonomy and intelligence into assistive machines---to offload some of the control burden from the user. Robots already synthetically sense, act in and reason about the world, and these technologies can be leveraged to help bridge the gap left by sensory, motor or cognitive impairments in the users of assistive machines. However, here the human-robot team is a very particular one: the robot is physically supporting or attached to the human, replacing or enhancing lost or diminished function. In this case getting the allocation of control between the human and robot right is absolutely essential, and will be critical for the adoption of physically assistive robots within larger society. This talk will overview some of the ongoing projects and studies in my lab, whose research lies at the intersection of artificial intelligence, rehabilitation robotics and machine learning. We are working with a range of hardware platforms, including smart wheelchairs and assistive robotic arms. A distinguishing theme present within many of our projects is that the machine automation is customizable---to a user's unique and changing physical abilities, personal preferences or even financial means.

Brenna Argall is the June and Donald Brewer Junior Professor of Electrical Engineering & Computer Science at Northwestern University, and also an assistant professor in the Department of Mechanical Engineering and the Department of Physical Medicine & Rehabilitation. Her research lies at the intersection of robotics, machine learning and human rehabilitation. She is director of the assistive & rehabilitation robotics laboratory (argallab) at the Rehabilitation Institute of Chicago (RIC), the premier rehabilitation hospital in the United States, and her lab's mission is to advance human ability through robotics autonomy. Argall is a 2016 recipient of the NSF CAREER award. Her Ph.D. in Robotics (2009) was received from the Robotics Institute at Carnegie Mellon University, as well as her M.S. in Robotics (2006) and B.S. in Mathematics (2002). Prior to joining Northwestern, she was a postdoctoral fellow (2009-2011) at the École Polytechnique Fédérale de Lausanne (EPFL), and prior to graduate school she held a Computational Biology position at the National Institutes of Health (NIH).

Faculty Host: Stephen Nuske

Achieving optimality while staying safe is one of the key problems that arise when planning under uncertainty. We specifically focus on path planning for aerial vehicles, where the uncertainties arise due to unobserved winds and other air traffic. A flight plan or a policy that doesn’t take into account such uncertainties can not only result in highly inefficient flight paths but can also jeopardize safety. In this talk, we will first focus on how to reduce uncertainty in wind predictions by using airplanes in flight as a large-scale sensor network. In particular, we explore how information from existing commercial aircraft on their normal business can be harnessed to observe and predict weather phenomena at a continental scale in greater detail that currently available. In the second part of the talk, we consider the problem of path planning under uncertain winds and traffic conditions. Specifically we propose planning algorithms that trade off exploration and exploitation in near-optimal manner and have appealing no-regret properties. Further, we will also discuss how Probabilistic Signal Temporal Logic (PrSTL) can be adapted to the robotic path planning problems in order to guarantee safety. We will present results from longitudinal real-world studies that demonstrate effectiveness of the framework.

Ashish Kapoor is a senior researcher at Microsoft Research, Redmond. Currently, his research focuses on Aerial Informatics and Robotics with an emphasis on building intelligent and autonomous flying agents that are safe and enable applications that can positively influence our society. The research builds upon cutting edge research in machine intelligence, robotics and human-centered computation in order to enable an entire fleet of flying robots that range from micro-UAVs to commercial jetliners. Various applications scenarios include Weather Sensing, Monitoring for Precision Agriculture, Safe Cyber-Physical Systems etc. Ashish received his PhD from MIT Media Laboratory in 2006. He also holds FAA Commercial Pilot certificate (SEL), FAA Flight Instructor certificate (Airplane Single Engine and Instrument Airplane) and is an avid amateur aircraft builder (see build blog).

Faculty Host: Louis-Philippe Morency

Reception follows at 5:00 pm in Newell-Simon 1513


Subscribe to RI