Overview

The ultimate goal of my research is to build intelligent and autonomous robots that think, behave and interact with the world in the way that human beings do, so that they can better serve, assist and collaborate with people in their daily lives across work, home and leisure.

The fundamental research question is to ensure those robots operate efficiently and safely in a human-involved environment. I am interested in addressing the microscopic aspect of the problem, e.g. the design of the behavior system (i.e. a mapping from observation to action) for single robot, as well as the macroscopic aspect of the problem, e.g. the analysis and validation of the human-robot system from a multi-agent perspective. Such human-robot system can be human-robot collaboration system in production lines, future transportation system with both automated and human-driven vehicles, or general cyber physical social system (CPSS).

Projects

2021~ Task agnostic real-time perception and control with few-shot cross-platform adaptation
Abstract

Abstract: Many industrial tasks nowadays require machines to be flexible, i.e., they should be able to 1) understand the changing environments and tasks and 2) generate corresponding actions in real time. For example, for machine tending, the manipulators should generate actions regarding the real-time configuration of the materials, which needs to be perceived online. Hence, it is important to enable real-time perception-action loops for these intelligent manipulators in these flexible tasks. However, it remains challenging to optimally and efficiently configure and adapt these perception-action loops, under changing environments and tasks. For different tasks and environments, the optimal configuration of the perception-action loops may vary significantly, e.g., the mounting location of the camera, the focus on the camera, and the optimal update frequency of the perception-action loop. Moreover, there are variations across different hardware platforms so that the optimal configuration for one platform may not be optimal for the other. To ensure optimality and consistency across different platforms, we will develop a task agnostic few-shot learning method that can 1) automatically calibrate the perception-action loop to optimize user specified objectives (e.g., minimizing cycle time, maximizing the task success rate); 2) monitor and adapt the system in real time if the environment changes to maintain optimality.

Sponsor: Siemens
2020~ Safe Uncaged Industrial Robots
Abstract | Publications

Abstract: Safe operation of intelligent robots in interactive environments depends on accurate prediction of others and consequent safe control of the ego robot. However, it remains challenging to 1) generate high-fidelity prediction of humans; 2) soundly verify the uncertainty associated with the prediction; and 3) incorporate the prediction and the verified uncertainty in the control of the ego robot. This project targets to address these three issues by incorporating recent progresses in 1) human motion prediction through imitation learning and online adaptation; 2) sound verification of deep neural networks; and 3) safe control of robot motion through the safe set algorithm. The work can be applied to human robot collaboration in production lines.

Sponsor: Ford Motor Company

Publications:
[J8] R. Liu, and C. Liu, "Human Motion Prediction Using Adaptable Recurrent Neural Networks and Inverse Kinematics," in IEEE Control System Letters, vol. 5, no. 5, pp. 1651-1656, Nov. 2021.
[W6] T. Wei, C. Liu, "Safe control with neural network dynamic models," in RSS Workshop on Robotics for People: Perspectives on Interaction, Learning and Safety, 2021.
[W5] R. Liu, C. Liu, "IADA: Iterative Adversarial Data Augmentation using Formal Verification and Expert Guidance," in ICML Workshop on Human In the Loop Learning, 2021.
T. Wei, C. Liu, "Online Verification of Deep Neural Networks under Domain or Weight Shift," arXiv: 2106.12732.
2020~ 2021 Hierarchical Motion Planning for Efficient and Provably Safe Human-Robot Interactions
Abstract | Publications

Abstract: Safe and efficient robot motion planning is critical to ensure desired human-robot interactions. However, there are very few methods that can comprehensively address uncertainties in human behaviors, robot model mismatch, robot computation limits, and measurement and actuation noises in an integrated planning theme. This proposal targets to develop a planning method that addresses all these challenges using hierarchical motion planning. A hierarchical planning theme has multiple planners running in parallel that use different dynamic models of the system and have different planning horizons, sampling frequencies, and replanning frequencies. It can generate trajectory plans from coarse to fine, and efficiently separate time-sensitive computation (e.g., real-time safety response) from time-insensitive computation (e.g., figuring out the reference trajectory towards the goal). A set of design principles will be developed to ensure sound safety guarantees as well as the performance of the hierarchical planner (e.g., stable and dead-lock free).

Sponsor: Amazon Research Award

Publications:
[C38] W. Zhao, T. He, C. Liu, “Model free safe control for zero-violation reinforcement learning,” in Conference on Robot Learning, 2021.
[C37] C. Noren, W. Zhao, C. Liu, "Safe Adaptation with Multiplicative Uncertainties Using Robust Safe Set Algorithm," in Modeling, Estimation, and Control Conference, 2021.
[W8] S. He, W. Zhao, C. Hu, Y. Zhu, C. Liu, "A hierarchical long short term safety framework for efficient robot manipulation under uncertainty" in MECC Workshop on Safe Control and Learning under Uncertainty, 2021.
2019~ 2021 6DoF Robot Assembly Station of Consumer Electronic Production
Abstract | Publications

Abstract: Assembly of consumer electronics in manufacturing is a time-consuming and labor-intensive task. Assembly is one of the most important robot application in computer, communication, consumer electronic (3C) production. Programing a traditional industrial robotic manufacturing system requires a significant amount of time and resources (and therefore investment). This makes it difficult for production line to switch from one product to the next in a cost-effective way. Because of that high up-front cost, the life cycle of a production line is 2-5 years. However, the accelerated pace of product innovation has reduced the life cycle of each product to 3 - 6 months. The change in production cost comes from 3 aspects: (i) new fixture design, (ii) system fine-tuning, (iii) system re-calibration. To make robots and solutions more competitive in this field, there is a need to develop a workstation that avoid these 3 steps, or minimize human configuration efforts to simplify assembly works. This is required to reduce the effective life cycle of a production line to match the ones of the products. The objective of this project is to develop new manipulation technology that enables automatic assembly of delicate parts onto PCB without the need of expensive re-programming of the robot. The plan is to leverage machine learning to help interpret the information from the various sensors (force feedback, visual sensors, etc.) and train a robotic system to adequately grab various components and insert these components into the pre-designated slots on a PCB.

Sponsor: Efort

Publications:
[C31] B. Niu*, C. Wang*, C. Liu, “Tolerance-guided Policy Learning for Adaptable and Transferrable Delicate Industrial Insertion,” in Conference on Robot Learning, 2020.
2019~ 2020 Automatic Onsite Polishing of Large Complex Surfaces by Real Time Planning and Control
Abstract | Video | Publications

Abstract: Polishing and grinding of metallic parts is an important manufacturing operation in many industrial applications. It remains challenging to polish large complex surfaces. Polishing is predominantly done manually which is very time consuming, expensive and more importantly, can be a safety hazard for human operators using hand-held devices. We propose the assembly of a robot capable of polishing complex and relatively large surface areas of complex free form fabricated parts. By developing this robot for a complex and potentially un-safe operation, this application will lower the technical, operational, and economic barriers for companies to adopt robotic technologies.

Sponsor: Advanced Robotics for Manufacturing Institute

Publications:
[C30] W. Zhao, S. He, C. Wen, and C. Liu. “Contact-Rich Trajectory Generation in Confined Environments Using Iterative Convex Optimization,” in ASME Dynamic Systems and Control Conference, 2020. Best Student Paper Finalist.
2019 Adaptable Behavior Prediction for Autonomous Driving
Abstract | Publications

Abstract: In highly interactive driving scenarios, accurate prediction of other road participants is critical for safe and efficient navigation of autonomous cars. Prediction is challenging due to the difficulty in modeling various driving behavior, or learning such a model. The model should be interactive and reflect individual differences. Imitation learning methods are able to learn interactive models. However, the learned models usually average out individual differences. When used to predict trajectories of individual vehicles, these models are biased. This project investigates adaptable prediction frameworks, which performs online adaptation of the offline learned models to recover individual differences and time-varying behaviors for better prediction. In particular, we combine a family of recursive least square parameter adaptation algorithms (RLS-PAA) with various offline learned models. RLS-PAA has analytical solutions and is able to adapt the model for every single vehicle efficiently online.

Sponsor: Holomatic

Publications:
[J7] A. Abuduweili, and C. Liu, “Robust Nonlinear Adaptation Algorithms for Multi-Task Prediction Networks,” in International Journal of Adaptive Control and Signal Processing, vol. 35, no. 3, pp. 314-341, Mar. 2021.
[C26] A. Abuduweili, and C. Liu, “Robust online model adaptation by extended kalman filter with exponential moving average and dynamic multi-epoch strategy,” in Learning for Dynamics and Control (L4DC) Conference, 2020.
[C21] W. Si, T. Wei, and C. Liu, "AGen: Adaptable generative prediction networks for autonomous driving", in IEEE Intelligent Vehicle Symposium, 2019.
2018~ 2021 Verification of Deep Neural Networks
Abstract | Publications | Code | Slides | Talk | Tutorial

Abstract: Deep neural networks are widely used for nonlinear function approximation with applications ranging from computer vision to control. Although these networks involve the composition of simple arithmetic operations, it can be very challenging to verify whether a particular network satisfies certain input-output properties. This project classifies methods that have emerged recently for soundly verifying such properties. These methods borrow insights from reachability analysis, optimization, and search. We investigate fundamental differences and connections between existing algorithms. In addition, we provide pedagogical implementations of existing methods and compare them on a set of benchmark problems. Moreover, we will extend these tools to verify closed-loop systems with NN components by combining with control-theoretic analysis.

Publications:
[J9] C. Liu, T. Arnon, C. Lazarus, C. Strong, C. Barrett, and M. Kochenderfer, "Algorithms for verifying deep neural networks", in Foundation and Trend in Optimization, vol. 4, No. 3-4, pp. 244–404, 2021.
[W1] C. Liu, T. Arnon, C. Lazarus, and M. Kochenderfer, "NeuralVerification.jl: Algorithms for verifying deep neural networks", in ICLR Debugging Machine Learning Models Workshop, 2019.
2018~ Micro to Macro Traffic Management and Modeling with Autonomous Vehicles
Abstract | Publications

Abstract: Automated vehicles are believed to be the key solution for future mobility. As more and more automated vehicles drive on public roads, they will interact with each other, and with other road participants such as human-driven vehicles and pedestrians. Those interactions will deeply change and redefine today’s transportation system. The fundamental question is: how can we achieve a safe and efficient transportation system through the design of the driving strategies for single automated vehicle? This project is aimed to develop an understanding on how the behavior design of single agent and their communication strategy may affect the overall multi-agent system, and how to achieve the best design from the system perspective. Indeed, the macro transportation system depends on the micro behaviors of road participants, while the micro behaviors of road participants are affected by others in the transportation system. From micro design to macro analysis, we are expected to gain better understandings of the micro-macro relationships and achieve a safe and efficient transportation system through the introduction of automated vehicles.

Publications:
C. Liu, and M. Kochenderfer, "Analyzing traffic delay at unmanaged intersections".
[J4] C. Liu, C. Lin, S. Shiraishi, and M. Tomizuka, "Distributed conflict resolution for connected autonomous vehicles", in IEEE Transactions on Intelligent Vehicles, 2018.
[C23] Y. Lin, H. Hsu, S. Lin, C. Lin, H. Jiang, and C. Liu, "Graph-Based Modeling, Scheduling, and Verification for Intersection Management of Intelligent Vehicles", in International Conference on Embedded Software, 2019.
[C17] C. Liu, and M. Kochenderfer, "Analytically modeling unmanaged intersections with microscopic vehicle interactions", in Intelligent Transportation Systems Conference (ITSC), 2018.
[C16] C. Liu, C. Lin, S. Shiraishi, and M. Tomizuka, "Improving efficiency of autonomous vehicles via V2V communication", in American Control Conference, 2018.
2017~ Safe and Efficient Robot Collaboration System (SERoCS)
Abstract | Link | Publications

Abstract: In factory automation, humans and robots comprise the two major work forces. Traditionally, humans and robots have not physically collaborated with each other during operation, in significant part because full automation with robots was the goal. In recent years, however, it has been recognized that there are tremendous advantages if robots are brought out of their cages and to allow them to share work space with and to collaborate with humans to take advantage of the best of two worlds – on one hand, the reliable execution of tasks by robots without wear handling objects of a wide range of sizes and weights, and on the other hand, the intelligence of humans and their five senses-based adaptability and flexibility. For collaboration between humans and robots to be successful, it is a prerequisite to ensure the safety of the humans in such collaboration. At the same time, it is important to ensure that robots collaborate with humans to ensure the best performance possible. The goal of this project is to establish a set of design principles of safe and efficient robot collaboration systems (SERoCS) for the next generation co-robots, which consists of robust cognition algorithms for environment monitoring, optimal task planning algorithms for safe human-robot collaboration, and safe motion planning and control algorithms for safe human-robot interactions (HRI). The proposed SERoCS will significantly expand the skill sets of the co-robots and prevent or minimize occurrences of human-robot collision and robot-robot collision during operation.

Publications:
C. Liu, T. Tang, H. Lin, Y. Cheng, and M. Tomizuka, "SERoCS: Safe and efficient robot collaborative systems for next generation intelligent industrial co-robots".
[C24] T. Wei, and C. Liu, "Safe control algorithms using energy functions: A unified framework, benchmark, and new directions", in IEEE Conference on Decision and Control, 2019.
[C20] Y. Cheng, W. Zhao, C. Liu, and M. Tomizuka, "Human motion prediction using semi-adaptable neural networks", in American Control Conference, 2019.
[C18] H. Lin, C. Liu, and M. Tomizuka, "Fast robot motion planning with collision avoidance and temporal optimization", in International Conference on Control, Automation, Robotics and Vision (ICARCV), 2018.
2014~ 2017 Robustly-Safe Automated Driving (ROAD) Systems
Abstract | Video | Poster | Link | Publications

Abstract: Automated driving is widely viewed as a promising technology to revolutionize today’s transportation system, so as to free the human drivers, ease the road congestion and lower the fuel consumption among other benefits. Substantial research efforts are directed into this field from research groups and companies. When the automated vehicles drive on public roads, they are automatically given social attributions. While existing technologies can assure high-fidelity sensing, real-time computation and robust control, the challenges lie in the interactions between the automated vehicle and the environment which includes other manually driven vehicles. We proposed a framework in designing the driving behavior for automated vehicles to prevent or minimize occurrences of collisions among vehicles and obstacles while maintaining efficiency (e.g. maintaining high speed on freeway).

Publications:
[C15] J. Chen, C. Liu, and M. Tomizuka, "FOAD: Fast optimization-based autonomous driving motion planner", in American Control Conference, 2018.
[C13] C. Liu, Y. Wang, and M. Tomizuka, "Boundary layer heuristic for search-based nonholonomic path planning in maze-like environments", in IEEE Intelligent Vechicle Symposium, 2017.
[C12] C. Liu, W. Zhan, and M. Tomizuka, "Speed profile planning in dynamic environments via temporal optimization", in IEEE Intelligent Vechicle Symposium, 2017.
[C11] W. Zhan, J. Chen, C. Chan, C. Liu, and M. Tomizuka, "Spatially-partitioned environmental representation and planning architecture for on-road autonomous driving", in IEEE Intelligent Vechicle Symposium, 2017.
[C9] C. Liu, J. Chen, T. Nguyen, and M. Tomizuka, "The robustly-safe automated driving system for enhanced active safety", in SAE World Congress, 2017.
[C8] W. Zhan, C. Liu, C. Chan, and M. Tomizuka, "A non-conservatively defensive strategy for urban autonomous driving", in IEEE Intelligent Transportation Systems Conference (ITSC), 2016.
[C6] C. Liu, and M. Tomizuka, "Enabling safe freeway driving for automated vehicles", in American Control Conference, 2016.
2013~ 2017 Robot Safe Interaction Systems (RSIS) for Intelligent Industrial Co-Robots
Abstract | Video | Talk | Publications

Abstract: With the development of modern robotics, robots are entering people's life in multiple ways. As identified by National Robotics Initiative (NRI), future intelligent robots can be co-defenders, co-explorers, co-inhabitants and even co-workers to human. To successfully launch those co-robots, we must make sure that they are safe to human users. However, this is not a easy task as the robots are operating in a dynamic uncertain environment (DUE) together with other intelligent agents such as humans. In this project, we address the safety issues in the context of (1) multi-agent interactions (2) sensing and knowledge representations (3) learning and predictions (4) human modeling and (5) constrained optimal control and decision-making. (Spotlight Talk in 2015 Bay Area Robotics Symposium.)

Publications:
C. Liu, and M. Tomizuka, "Robot Safe Interaction Systems for Intelligent Industrial Co-Robots".
[J3] C. Liu, C. Lin, and M. Tomizuka, "The convex feasible set algorithm for real time optimization in motion planning", in SIAM Journal on Control and Optimization, 2018.
[J2] C. Liu, and M. Tomizuka, "Real time trajectory optimization for nonlinear robotic systems: Relaxation and convexification", in System & Control Letters, 2017.
[B1] C. Liu, and M. Tomizuka, "Designing the robot behavior for safe human-robot interactions", in Trends in Control and Decision-Making for Human-Robot Collaboration Systems (Y. Wang and F. Zhang (Eds.)), Springer, 2017.
[C14] C. Lin, C. Liu, Y. Fan, and M. Tomizuka, "Real-time collision avoidance algorithm on industrial manipulators", in IEEE Conference on Control Technology and Applications (CCTA), 2017.
[C10] C. Liu, C. Lin, Y. Wang, and M. Tomizuka, "Convex feasible set algorithm for constrained trajectory smoothing", in American Control Conference, 2017.
[C4] C. Liu, and M. Tomizuka, "Algorithmic safety measures for intelligent industrial co-robots", in IEEE International Conference on Robotics and Automation (ICRA), 2016.
[C3] C. Liu, and M. Tomizuka, "Safe exploration: addressing various uncertainty levels in human robot interactions", in American Control Conference, 2015.
[C2] C. Liu, and M. Tomizuka, "Control in a safe set: addressing safety in human-robot interactions", in ASME Dynamic Systems and Control Conference, 2014. Best Student Paper Finalist.
[C1] C. Liu, and M. Tomizuka, "Modeling and controller design of cooperative robots in workspace sharing human-robot assembly teams", in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2014.