**UNDER CONSTRUCTION: When I say it is under construnction, I really mean it.**

Bot Intelligence Group (BIG)
Bring your own bot, and we will add intelligence to them
Members | Bots | Projects | Publications

In our group, we develop algorithms and intelligence/learning architectures to provide cognitive capabilities for robots.

Keywords: conversational mobile robots, social navigation, planning from natural language, vision-language fusion, semantic classification, image to text, text to image, plan to explanation; probabilistic models, reinforcement learning, inverse reinfocement learning (IRL), deep IRL, deep learning

Jean Oh, PI Xavier Perez Andy (Chieh-En) Tsai Junjiao Tian Yeeho Song
Satyen Rajpal Krishna Toshniwal Jagjeet Singh Shuhan Yang Balarama Buddharaju
Brandon Trabucco Pranav Dheram Chris Hazard Kai-Chi Huang

Meet our robots that can follow natural language commands.
(Ballbot by Ralph Hollis)


Conversational Mobile Robots

S.-R. Shiang, A. Gershman, and J. Oh   A Generalized Model for Multimodal Perception. AAAI Fall Symposium, November 2017 [pdf].

J. Hu, D. Fan, S. Yao, and J. Oh   Answer-Aware Attention on Grounded Question Answering in Images. AAAI Fall Symposium, November 2017 [pdf].

S. Shiang, S. Rosenthal, A. Gershman, J. Carbonell, J. Oh. Vision-Language Fusion for Object Recognition.  In Proc. of AAAI Conference on Artificial Intelligence (AAAI), 2017. [pdf]

J. Hu, J. Oh, A. Gershman. Learning Lexical Entries for Robotic Commands using Crowdsourcing. In Proc. of AAAI Conference on Human Computation (HCOMP), 2016 (short paper).[pdf]

J. Oh, M. Zhu, S. Park, T.M. Howard, M.R. Walter, D. Barber, O. Romero, A. Suppe, L. Navarro-Serment, F. Duvallet, A. Boularias, J. Vinokurov, T. Keegan, R. Dean, C. Lennon, B. Bodt, M. Childers, J. Shi, K. Daniilidis, N. Roy, C. Lebiere, M. Hebert, and A. Stentz. Integrated intelligence for human-robot teams. In Proc. of International Symposium on Experimental Robotics (ISER) 2016 [pdf].

A. Boularias, F. Duvallet, J. Oh, and A. Stentz. Learning Qualitative Spatial Relations for Robotic Navigation. in  Proc. of International Joint Conference on Artificial Intelligence (IJCAI), 2016 [pdf].

J. Oh, L. Navarro-Serment, A. Suppe, A. Stentz, M. Hebert, Inferring door locations from a teammate's trajectory in stealth human-robot team operations. In Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015 [pdf].

J. Oh, A. Suppe, F. Duvallet, A. Boularias, J. Vinokurov, L. Navarro-Serment, O. Romero, R. Dean, C. Lebiere, M. Hebert, and A. Stentz. Toward mobile robots reasoning like humans. In Proc. of AAAI Conference on Artificial Intelligence (AAAI), 2015. [pdf]

A. Boularias, F. Duvallet, J. Oh, and A. Stentz. Learning to ground spatial relations for outdoor robot navigation. In Proc. of IEEE Conference on Robotics and Automation (ICRA), 2015. [pdf] *Best Cognitive Robotics Paper Award*

Social Navigation

A. Vemula, K. Muelling, J. Oh. Social Attention: Modeling Attention in Human Crowds. In Proc. of IEEE Conference on Robotics and Automation (ICRA), 2018. *Best Paper Award in Cognitive Robotics* [ArXiv].

A. Vemula, K. Muelling, J. Oh. Modeling cooperative navigation in dense human crowds. In Proc. of IEEE Conference on Robotics and Automation (ICRA), 2017 [pdf].

A. Vemula, K. Muelling, J. Oh. Path Planning in Dynamic Environments with Adaptive Dimensionality. In Proc. of International Symposium on Combinatorial Search (SoCS) 2016. [pdf]

Fast & Frugal Semantic Classification


Deep Inverse Reinforcement Learning for Safe Navigation


 Team pictures
BIG 2018Robot Autonomy, Project Andy team, 2016
Sz-Rung ShiangLTI MSBloomberg Research, NYC
Junjie HuLTI MSLTI PhD student
Anirudh VemulaRI MS RI PhD student
Matthew WilsonRISS Summer ScholarUniv. of Utah undergraduate student
Jie-Eun HwangVisiting scholarProfessor, Univ. of Seoul, Korea
Kevin Zhang Summer intern CMU graduate student
Michael Jason Gnanasekar Summer intern Facebook
Sonia Appasamy Summer intern Cornell undergraduate student

Acknowledgement: Our research is funded by U.S. Army Research Lab., Air Force Office of Scientific Research, Defense Advanced Research Projects Agency, and DiDi Chuxing.