**UNDER CONSTRUCTION: When I say it is under construnction, I really mean it.**

Bot Intelligence Group (BIG)
Bring your own bot, and we will add intelligence to them
Members | Bots | Projects | Publications

In our group, we develop algorithms and intelligence/learning architectures to provide cognitive capabilities for robots.

Keywords: conversational mobile robots, social navigation, planning from natural language, vision-language fusion, semantic classification, image to text, text to image, plan to explanation; probabilistic models, reinforcement learning, inverse reinfocement learning (IRL), deep IRL, deep learning

Jean Oh, PI Xavier Perez Andy (Chieh-En) Tsai Junjiao Tian Yeeho Song
Satyen Rajpal Krishna Toshniwal Jagjeet Singh Shuhan Yang Balarama Buddharaju
Brandon Trabucco Pranav Dheram Chris Hazard

Meet our robots that can follow natural language commands.
(Ballbot by Ralph Hollis)


Conversational Mobile Robots

S.-R. Shiang, A. Gershman, and J. Oh   A Generalized Model for Multimodal Perception. AAAI Fall Symposium, November 2017 [pdf].

J. Hu, D. Fan, S. Yao, and J. Oh   Answer-Aware Attention on Grounded Question Answering in Images. AAAI Fall Symposium, November 2017 [pdf].

S. Shiang, S. Rosenthal, A. Gershman, J. Carbonell, J. Oh. Vision-Language Fusion for Object Recognition.  In Proc. of AAAI Conference on Artificial Intelligence (AAAI), 2017. [pdf]

J. Hu, J. Oh, A. Gershman. Learning Lexical Entries for Robotic Commands using Crowdsourcing. In Proc. of AAAI Conference on Human Computation (HCOMP), 2016 (short paper).[pdf]

J. Oh, M. Zhu, S. Park, T.M. Howard, M.R. Walter, D. Barber, O. Romero, A. Suppe, L. Navarro-Serment, F. Duvallet, A. Boularias, J. Vinokurov, T. Keegan, R. Dean, C. Lennon, B. Bodt, M. Childers, J. Shi, K. Daniilidis, N. Roy, C. Lebiere, M. Hebert, and A. Stentz. Integrated intelligence for human-robot teams. In Proc. of International Symposium on Experimental Robotics (ISER) 2016 [pdf].

A. Boularias, F. Duvallet, J. Oh, and A. Stentz. Learning Qualitative Spatial Relations for Robotic Navigation. in  Proc. of International Joint Conference on Artificial Intelligence (IJCAI), 2016 [pdf].

J. Oh, L. Navarro-Serment, A. Suppe, A. Stentz, M. Hebert, Inferring door locations from a teammate's trajectory in stealth human-robot team operations. In Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015 [pdf].

J. Oh, A. Suppe, F. Duvallet, A. Boularias, J. Vinokurov, L. Navarro-Serment, O. Romero, R. Dean, C. Lebiere, M. Hebert, and A. Stentz. Toward mobile robots reasoning like humans. In Proc. of AAAI Conference on Artificial Intelligence (AAAI), 2015. [pdf]

A. Boularias, F. Duvallet, J. Oh, and A. Stentz. Learning to ground spatial relations for outdoor robot navigation. In Proc. of IEEE Conference on Robotics and Automation (ICRA), 2015. [pdf] *Best Cognitive Robotics Paper Award*

Social Navigation

A. Vemula, K. Muelling, J. Oh. Social Attention: Modeling Attention in Human Crowds. In Proc. of IEEE Conference on Robotics and Automation (ICRA), 2018. *Best Paper Award in Cognitive Robotics* [ArXiv].

A. Vemula, K. Muelling, J. Oh. Modeling cooperative navigation in dense human crowds. In Proc. of IEEE Conference on Robotics and Automation (ICRA), 2017 [pdf].

A. Vemula, K. Muelling, J. Oh. Path Planning in Dynamic Environments with Adaptive Dimensionality. In Proc. of International Symposium on Combinatorial Search (SoCS) 2016. [pdf]

Fast & Frugal Semantic Classification


Deep Inverse Reinforcement Learning for Safe Navigation


Sz-Rung ShiangLTI MSBloomberg Research, NYC
Junjie HuLTI MSLTI PhD student
Anirudh VemulaRI MS RI PhD student
Matthew WilsonRISS Summer ScholarUniv. of Utah undergraduate student
Jie-Eun HwangVisiting scholarProfessor, Univ. of Seoul, Korea
Kevin Zhang Summer intern CMU graduate student
Michael Jason Gnanasekar Summer intern Facebook
Sonia Appasamy Summer intern Cornell undergraduate student

Acknowledgement: Our research is funded by U.S. Army Research Lab., Air Force Office of Scientific Research, Defense Advanced Research Projects Agency, and DiDi Chuxing.