Object Goal Navigation using
Goal-Oriented Semantic Exploration

Devendra Singh Chaplot
Dhiraj Gandhi
Abhinav Gupta
Ruslan Salakhutdinov
CMU
FAIR
CMU/FAIR
CMU
Accepted at NeurIPS, 2020
Winner of the CVPR 2020 Habitat ObjectNav Challenge

[Paper]
[Talk]
[Slides]



This work studies the problem of object goal navigation which involves navigating to an instance of the given object category in unseen environments. End-to-end learning-based navigation methods struggle at this task as they are ineffective at exploration and long-term planning. We propose a modular system called, 'Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category. Empirical results in visually realistic simulation environments show that the proposed model outperforms a wide range of baselines including end-to-end learning-based methods as well as modular map-based methods and led to the winning entry of the CVPR-2020 Habitat ObjectNav Challenge. Ablation analysis indicates that the proposed model learns semantic priors of the relative arrangement of objects in a scene, and uses them to explore efficiently. Domain-agnostic module design allow us to transfer our model to a mobile robot platform and achieve similar performance for object goal navigation in the real-world.


Goal-Oriented Semantic Exploration

The proposed model consists of two modules, Semantic Mapping and Goal-Oriented Semantic Policy. The Semantic Mapping module builds a semantic map over time and the Goal-Oriented Semantic Policy selects a long-term goal based on the semantic map to reach the given object goal efficiently. A deterministic local policy based on analytical planners is used to take low-level navigation actions to reach the long-term goal.




Short Presentation

A short presentation at the CVPR 2020 Embodied AI Workshop describing our winning entry to the Habitat ObjectNav Challenge.



Demo Video



Real-World Transfer




Media

A short video created by CMU Media team describing the key ideas behind this project.

Read more about the project in the following media articles:


Paper and Bibtex

[Paper]

Citation
 
Chaplot, D.S., Gandhi, D., Gupta, A. and Salakhutdinov, R. 2020.
Object Goal Navigation using Goal-Oriented Semantic Exploration.
In Neural Information Processing Systems (NeurIPS-20).

[Bibtex]
@inproceedings{chaplot2020object,
  title={Object Goal Navigation using Goal-Oriented Semantic Exploration},
  author={Chaplot, Devendra Singh and Gandhi, Dhiraj and
            Gupta, Abhinav and Salakhutdinov, Ruslan},
  booktitle={In Neural Information Processing Systems}
  year={2020}}
                


Related Projects



Acknowledgements

This work was supported by IARPA DIVA D17PC00340, US Army W911NF1920104, ONR Grant N000141812861, ONR MURI, ONR Young Investigator, DARPA MCS and Apple. We would also like to acknowledge NVIDIA’s GPU support.
Website template from here and here.