Liu Ren, Ph.D.

Contact info: williamren at gmail dot com

Liu Ren is the VP and Chief Scientist at Bosch Research North America and Bosch Center for Artificial Intelligence. He leads the development of cutting-edge AI technologies that integrate human and machine intelligence to drive innovation and growth for Robert Bosch Corporation, a global enterprise with over $90 billion in revenue and more than 400,000 employees worldwide. 

As the responsible global head, Liu oversees research activities conducted by several AI research departments and teams located in the Silicon Valley, Pittsburgh, Bangalore, and Renningen. His teams focus on a diverse range of AI research topics, including Big Data Visual Analytics, Explainable AI (XAI), Natural Language Processing (NLP), Robotics, 3D Perception and Mixed Reality, Audio Analytics, and AI platforms for industrial AI application areas such as autonomous driving, advanced driver assistance systems (ADAS), Industry 4.0 or smart manufacturing, and consumer goods such as home appliances. Liu's technical expertise and leadership have also been recognized with multiple awards, including the Bosch North America Inventor of the Year Award for 3D mapping in 2016 and the Best Paper Award (2018, 2020) and Best Paper Honorable Mention Award (2016, 2022) for big data visual analytics and XAI in IEEE Visualization Conference. He has also successfully led the development of several key AI and HMI products, published more than 40 papers in top-tier AI and CS conferences and journals, and holds more than 50 patents.

Liu received his Ph.D. and M.Sc. degrees in Computer Science from Carnegie Mellon University's Computer Science Department and earned his B.Sc. in Computer Science from Zhejiang University, China, after studying in the elite Chu Kochen Honors College program there.

Selected Publications (Full list: Google Scholar)

Wenbin He, Suphanut Jamonnak, Liang Gou, Liu Ren. “CLIP-S4: Language-Guided Self-Supervised Semantic Segmentation”, The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 (highlight)

Nathaniel Merrill, Yuliang Guo, Xingxing Zuo, Xinyu Huang, Stefan Leutenegger, Xi Peng, Liu Ren, and Guoquan Huang. "Symmetry and Uncertainty-Aware Object SLAM for 6dof Object Pose Estimation", IEEE/CVF Conference on Computer Vision and Pattern Recognition (IEEE CVPR 2022).

Yuyan Li, Yuliang Guo, Zhixin Yan, Xinyu Huang, Ye Duan, and Liu Ren. "Omnifusion: 360 Monocular Depth Estimation via Geometry-Aware Fusion", video link, IEEE/CVF Conference on Computer Vision and Pattern Recognition (IEEE CVPR Oral 2022). 

Xiaoyu Zhang, Jorge Piazentin Ono, Huan Song, Liang Gou, Kwan-Liu Ma, and Liu Ren. "SliceTeller: A Data Slice-Driven Approach for Machine Learning Model Validation.", short video link, long video link, IEEE Transactions on Visualization and Computer Graphics (IEEE VIS 2022, Best Paper Honorable Mention Award)

Md Naimul Hoque, Wenbin He, Arvind Kumar Shekar, Liang Gou, Liu Ren. “Visual Concept Programming: A Visual Analytics Approach to Injecting Human Intelligence at Scale”, video link, IEEE Transactions on Visualization and Computer Graphics (IEEE VIS 2022)

Wenbin He, William Surmeier, Arvind Kumar Shekar, Liang Gou, Liu Ren, “Self-supervised Semantic Segmentation Grounded in Visual Concepts”, International Joint Conference on Artificial Intelligence (IJCAI 2022)

Arvind Kumar Shekar, Liang Gou, Liu Ren, Axel Wendt, “Label-free robustness estimation of object detection CNNs for autonomous driving applications”,  International Journal of Computer Vision (IJCV 2022)

Sascha Hornauer, Ke Li, X Yu Stella, Shabnam Ghaffarzadegan, Liu Ren, “Unsupervised Discriminative Learning of Sounds for Audio Event Classification”, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2021)

Zhenge Zhao, Panpan Xu, Carlos Scheidegger, Liu Ren, “Human-in-the-loop Extraction of Interpretable Concepts in Deep Learning Models”, video link, IEEE Transactions on Visualization and Computer Graphics (IEEE VAST 2021)

Huan Song, Zeng Dai, Panpan Xu, Liu Ren, “Interactive Visual Pattern Search on Graph Data via Graph Representation Learning”, IEEE Transactions on Visualization and Computer Graphics (IEEE VAST 2021)

Wenbin He, Lincan Zou, Arvind Kumar Shekar, Liang Gou, Liu Ren, “Where Can We Help? A Visual Analytics Approach to Diagnosing and Improving Semantic Segmentation of Movable Objects”, video link,  IEEE Transactions on Visualization and Computer Graphics (IEEE VAST 2021)

Jason Zink, Ryan Todd, Manjunath Kumbar, Arun Biyani, Xinyu Huang, and Liu Ren, "Virtual Visor: Adding Intelligence to LCD Displays to Selectively Block Sunlight", video link, SID International Conference on Display Technology (SID ICDT 2021).

Kaiqiang Song, Bingqing Wang, Zhe Feng, Liu Ren, and Fei Liu. “Controlling the Amount of Verbatim Copying in Abstractive Summarization”, The AAAI Conference on Artificial Intelligence (AAAI 2020).

Liang Gou, Lincan Zou, Nanxiang Li, Michael Hofmann, Arvind Kumar Shekar, Axel Windt, Liu Ren, "VATLD: A Visual Analytics System to Assess, Understand and Improve Traffic Light Detection", video,  IEEE Transactions on Visualization and Computer Graphics (IEEE VAST 2020). Best Paper Award

Summary: A first visual analytics approach to addressing the very important AI function testing issues (e.g., traffic light detection) in autonomous driving by leveraging semantic representation learning and adversarial learning with a minimal human-in-the-loop strategy.  

Junhan Zhao, Zeng Dai, Panpan Xu, Liu Ren, “ProtoViewer: Visual Interpretation and Diagnostics of Deep Neural Networks with Factorized Prototypes”, IEEE Transactions on Visualization and Computer Graphics (IEEE VAST 2020)

Yao Ming, Panpan Xu, Furui Cheng, Huamin Qu, Liu Ren, "ProtoSteer: Steering Deep Sequence Model with Prototypes", video,  IEEE Transactions on Visualization and Computer Graphics 2020 (IEEE VAST 2019)

Summary: ProtoSteer enabling experts to inspect, critique, and revise explainable AI (XAI) models  (represented as a small set of prototypes) interactively with domain know-how for different AI applications (e.g., NLP,  predictive diagnostics).

Yulin Yang, Benzun Pious Wisely Babu, Chuchu Chen, Guoquan Huang, Liu Ren, "Analytic Combined IMU Integration (ACI^2) For Visual Inertial Navigation", 2020 IEEE International Conference on Robotics and Automation (ICRA 2020)

Summary: Addressing key performance bottlenecks in visual-inertial sensor fusion: a modularized analytic combined IMU integrator (ACI^2) with elegant derivations for IMU integrations, bias Jabcobians and related covariances. 

Yao Ming, Panpan Xu, Huamin Qu, Liu Ren, "Interpretable and Steerable Sequence Learning via Prototypes", video, ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM KDD 2019), Research Track, Oral 

Summary: A novel interpretable and steerable deep sequence model (ProSeNet) for explainable AI with natural explanations derived from case-based reasoning. 

Benzun Pious Wisely Babu, Zhixin Yan, Mao Ye, Liu Ren, "On Exploiting Per-Pixel Motion Conflicts to Extract Secondary Motions", video,IEEE International Symposium on Mixed and Augmented Reality 2017 (ISMAR 2018)

Summary: A novel approach addressing motion conclit problems in visual-inertial sensor fusion, enabling better augmentation of virtual content attached to secondary motions for Ubiquitous Augmented Reality (AR).

Dongyu Liu, Panpan Xu, Liu Ren, "TPFlow: Progressive Partition and Multidimensional Pattern Extraction for Large-Scale Spatio-Temporal Data Analysis", video,  IEEE Transactions on Visualization and Computer Graphics (IEEE VAST 2018) Best Paper Award

Summary: A first visual analytics solution to handle multidimensional (>2D) spatial temporal data analysis for easy extraction and intuitive visualization of latent patterns based on a novel piecewise rank-one tensor decomposition algorithm.

Gromit Yeuk-Yin Chan, Panpan Xu, Zeng Dai, Liu Ren, "ViBR: Visualizing Bipartite Relations at Scale with the Minimum Description Length Principle", video, IEEE Transactions on Visualization and Computer Graphics (IEEE VAST 2018)

 Summary: A novel visual summarization technique for interactive analysis of large bipartite graphs based on the minimum description length (MDL) principle and locality sensitive hashing (LSH). 

Zhixin Yan, Mao Ye, and Liu Ren, "Dense Visual SLAM with Probabilistic Surfel Map", video, IEEE International Symposium on Mixed and Augmented Reality 2017 (ISMAR 2017), and selected for TVCG publication (IEEE Transactions on Visualization and Computer Graphics).

Summary: A novel map representation called Probabilistic Surfel Map (PSM) that provides globally consistent map of the environment for dense visual SLAM, leading to a drastic performance improvement (e.g, tracking errors) compared to the state of the art approach (e.g, σ-DVO

Alsallakh Bilal, Amin Jourabloo, Mao Ye, Xiaoming Liu, and Liu Ren, "Do Convolutional Neural Networks Learn Class Hierarchy?", video, IEEE Transactions on Visualization and Computer Graphics (IEEE VAST 2017)

Summary: A novel explainable AI approach that can help to improve the performance of general CNN-based classifiers with ease by leveraging visual analytics to improve CNN model structure and identify issues in the training data 

Yuanzhe Chen, Panpan Xu, and Liu Ren, "Sequence Synopsis: Optimize Visual Summary of Temporal Event Data", video, supplementary material,  IEEE Transactions on Visualization and Computer Graphics (IEEE VAST 2017).

Summary: A novel event sequence summarization and visualization approach based on the minimum description length (MDL) principle addressing several key AI application areas (e.g., predictive diagnostics for connected vehicles)

Amin Jourabloo,  Xiaoming Liu,  Mao Ye, and  Liu Ren, "Pose-Invariant Face Alignment with a Single CNN",  video,  International Conference on Computer Vision (ICCV 2017).

Summary:  A novel large-pose face alignment method with fast end-to-end training in a single CNN

Panpan Xu, Honghui Mei, Liu Ren,  and Wei Chen, “ViDX: Visual Diagnostics of Assembly Line Performance in Smart Factories”,   video, IEEE Transactions on Visualization and Computer Graphics (IEEE VAST 2016). Best Paper Honorable Mention Award

Summary: A first visual analytics approach to addressing the needs (e.g., easy process optimization and intuitive troubleshooting for manufacturing)  of a new application domain-Industry 4.0 or Connected Industry 

Bilal Alsallakh and Liu Ren, “PowerSet: A Comprehensive Visualization of Set Intersections”, video, IEEE Transactions on Visualization and Computer Graphics (IEEE Info VIS 2016).

Summary: A simple and novel set visualization approach for various kinds of big data analytics applications (e.g., predictive analytics) 

Chao Du, Yen-Lin Chen, Mao Ye, and Liu Ren, “Edge Snapping-Based Depth Enhancement for Dynamic Occlusion Handling in Augmented Reality”,  video, IEEE International Symposium on Mixed and Augmented Reality 2016,  (full paper for ISMAR 2016).

Summary: A near real-time approach to dealing with dynamic occlusion handling challenges with high quality visual presentations for both video see-through and optics see-through AR applications  

Benzun Wisely Babu, Soohwan Kim, Zhixin Yan, and Liu Ren, “σ-DVO: Sensor Noise Model Meets Dense Visual Odometry”, video, IEEE International Symposium on Mixed and Augmented Reality 2016,  (full paper for ISMAR 2016).

Summary: A novel dense tracking approach by incorporating uncertainty modeling of depth measurements in the optimization framework of RGBD sensing for AR,  leading to a drastic 25% error reduction compared to the state-of-the art solution 

Jennifer Chandler, Lei Yang and Liu Ren, "Procedural Window Lighting Effects for Real-Time City Rendering", video, ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D) 2015.

Summary: A fast and scalable approach to addressing challenges in large scale  night view rendering for big cities with applications in computer graphics and real-time geo-visualization for future infotainment systems

Mao Ye, Xianwang Wang, Ruigang Yang, Liu Ren, and Marc Pollefeys, "Accurate 3D Body Pose Estimation From a Single Depth Image", Proceedings of International Conference on Computer Vision (ICCV 2011), November, 2011 

Summary: A high accurate human pose estimation algorithm for entertainment applications that employ a depth sensor as input

Xinyu Huang, Liu Ren, and Ruigang Yang, "Image Deblurring for Less Intrusive Iris Capture",  IEEE Computer Society Coference on Computer Vision and Pattern Recognition (CVPR 2009), June, 2009

Summary: A long range and non-intrusive iris capture and recognition system featuring a novel image deblurring algorithm to handle the limitation of low cost hardware (patent granted)

Liu Ren, Alton Patrick, Alexei Efros, Jessica Hodgins, and James Rehg, "A Data-Driven Approach to Quantifying Natural Human Motion", ACM Transactions on Graphics (SIGGRAPH 2005),  August, 2005.

Summary: The first approach to automatic human animation quality evaluation


Liu Ren, Gregory Sharknarovich, Jessica Hodgins, Hanspeter Pfister and Paul Viola, "Learning Silhouette Features for Control of Human Motion", ACM Transactions on Graphics (SIGGRAPH 2004),  October, 2005.

Summary: A novel and low cost vision-based interface for "do-as-I-do" applications in entertainment industry

Wei Chen, Liu Ren, Matthias Zwicker and Hanspeter Pfister, "Hardware-Accelerated Adpative EWA Volume Splatting", IEEE Visualization 2004, October, 2004.

Summary: The first GPU-based approach to high quality volume splatting with EWA filtering (patent granted)

Liu Ren, Hanspeter Pfister and Matthias Zwicker, "Object Space EWA Surface Splatting: A Hardware Accelerated Approach to High Quality Point Rendering",  Computer Graphics Forum 21(3)( EUROGRAPHICS 2002, Best Paper Nominee), September, 2002.

Summary: The first GPU-based approach to high quality point-based rendering with EWA filtering (patent granted)

Selected Products 

3D artMap (watercolor & pencil hatching style, Berlin and Stuttgart)

3D artMap is the world's first stylized 3D navigation map powered by non-photorealistic (artistic ) rendering technologies.  3D artMap is the key feature in Robert Bosch's first iPhone app product (successfully launched in 2010.12 for 19 european countries) and several in-car navigation products. 

Press releases:

Automotive IT news [European Market]

Sina [Chinese Market]