About me

I'm Nick Rhinehart, a Ph.D Student at the Robotics Institute within the School of Computer Science at Carnegie Mellon University.

For systems to be generally intelligent, they must be able to reason about the future—
How should we learn, interpret, quantify, and leverage models that reason about the future?


Towards this question and others, I work on RL and IL methods at the interface of Computer Vision and Machine Learning. I'm specifically interested in building decision-theoretic models that leverage rich perception sources to inform forecasting and control tasks. I currently collaborate with Kris Kitani, Sergey Levine, Paul Vernaza, and Drew Bagnell.

In the past, I've worked with Sergey Levine at UC Berkeley, Paul Vernaza and Manmohan Chandraker at NEC Labs America, Yoichi Sato and Ryo Yonetani at The University of Tokyo, and Drew Bagnell at Uber ATG. I graduated from Swarthmore College with a degree in CS and a degree in Engineering. At Swarthmore I worked with Matt Zucker.



News

Publications

PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings

N. Rhinehart, R. McAllister, K. M. Kitani, S. Levine

Best Paper, ICML 2019 Workshop on AI for Autonomous Driving
ICCV 2019 (to appear) | pdf | show abs | show bib | project page | visualization code

Mini abstract: "We perform deep conditional forecasting with multiple interacting agents: when you control one of them, you can use its goals to better predict what nearby agents will do. The model also outperforms S.O.T.A. methods on the more standard task of unconditional forecasting."



	  		
	  	      

Lorem Ipsum
	  	        

Generative Hybrid Representations for Activity Forecasting with No-Regret Learning

J. Guan, Y. Yuan, K. M. Kitani, N. Rhinehart

arXiv 2019 | pdf | show abs | show bib

Mini abstract: "Some activites are best represented discretely, others continuously. We learn a deep likelihood-based generative model to jointly forecast discrete and continuous activities, and show how to tweak the model to learn efficiently online."



			
		      

Lorem Ipsum
		        

Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information

M. Sharma, A. Sharma, N. Rhinehart, K. M. Kitani

ICLR 2019 | pdf | show abs | show bib | project page

Mini abstract: "Many behaviors are naturally composed of sub-tasks. Our approach learns to imitate behaviors with subtasks by discovering topics of latent behavior to influence its imitation."



			
		      

Lorem Ipsum
			

Deep Imitative Models for Flexible Inference, Planning, and Control

N. Rhinehart, R. McAllister, S. Levine

Contributed talk at NeurIPS 2018 MLITS Workshop (top 3 of 25 papers)
Contributed talk at NeurIPS 2018 Infer2Control Workshop (top 6 of 27 papers)
arXiv 2018 | pdf | show abs | show bib | project page

Mini abstract: "We learn a deep conditional distribution of human driving behavior to guide planning and control of an autonomous car in simulation, without any trial-and-error data. We show that the approach can be adapted to execute tasks that were never demonstrated, including safely avoiding potholes, and is robust to misspecified goals that would cause it to violate its model of the rules of the road, and achieve S.O.T.A. on the CARLA benchmark."



			
		      

Lorem Ipsum
			

First-Person Activity Forecasting from Video with Online Inverse Reinforcement Learning

N. Rhinehart, K. Kitani

TPAMI 2018 | pdf | show abs | show bib | project page

Mini abstract: "We continuously model and forecast long-term goals of a first-person camera wearer through our Online Inverse RL algorithm. We show our approach learns efficiently continuously in theory and practice."



			    
			  

Lorem Ipsum
			    

R2P2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting

N. Rhinehart, K. M. Kitani, P. Vernaza

ECCV 2018 | pdf | show abs | show bib | project page | supplement | blog post | dataset soon

Mini abstract: "We designed an objective to jointly maximize diversity and precision for generative models, and designed a deep autoregressive flow to efficiently optimize this objective for the task of motion forecasting. Unlike many popular generative models, ours can exactly evaluate its probability density function for arbitrary points."



			
		      

Lorem Ipsum
			
			
			

Learning Neural Parsers with Deterministic Differentiable Imitation Learning

T. Shankar, N. Rhinehart, K. Muelling, K. M. Kitani

CORL 2018 | pdf | show abs | show bib | code

Mini abstract: "We developed and applied a new imitation learning approach for the task of sequential visual parsing."



			  
		        

Lorem Ipsum
		          

Human-Interactive Subgoal Supervision for Efficient Inverse Reinforcement Learning

X. Pan, E. Ohn-Bar, N. Rhinehart, Y. Xu, Y. Shen, K. M. Kitani

AAMAS 2018 | pdf | show abs | show bib

Mini abstract: "We analyze the benefit of incorporating a notion of subgoals into Inverse Reinforcement Learning (IRL) with a Human-In-The-Loop (HITL) framework and find our approach to require less demonstration data than a baseline Inverse RL approach"



		      
		    

Lorem Ipsum
		      

N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning

A. Ashok, N. Rhinehart, F. Beainy, K. Kitani

ICLR 2018 | pdf | show abs | show bib | code

Mini abstract: "We designed a principled method to perform neural model compression: we trained a compression agent via RL on the sequential task of compressing large networks while maintaining high performance. The compressing agent was able to generalize to compress previously-unseen networks."



		      
		    

Lorem Ipsum
		      

Predictive-State Decoders: Encoding the Future Into Recurrent Neural Networks

N. Rhinehart*, A. Venkataraman*, W. Sun, L. Pinto, M. Hebert, B. Boots, K. Kitani, J. A. Bagnell

NIPS 2017 | pdf | show abs | show bib

Mini abstract: "We use the idea of Predictive State Representations to guide learning of RNNs: by encouraging the hidden-state of the RNN to be predictive of future observations, we found it to improve RNN performance on various tasks in probabilistic filtering, imitation learning, and reinforcement learning."



		        
		      

Lorem Ipsum
		        

First-Person Activity Forecasting with Online Inverse Reinforcement Learning

N. Rhinehart, K. Kitani

Best Paper Honorable Mention (3 of 2,143 submissions)
ICCV 2017 | pdf | show abs | show bib | project page | code

Mini abstract: "We continuously model and forecast long-term goals of a first-person camera wearer through our Online Inverse RL algorithm. In contrast to motion forecasting, our approach reasons about semantic states and future goals that are potentially far away in space and time."



			    
			  

Lorem Ipsum
			    
Learning Action Maps of Large Environments Via First-Person Vision

N. Rhinehart, K. Kitani

CVPR 2016 | pdf | show abs | show bib

Mini abstract: "We developed an approach that learns to associate visual cues associated with sparse behaviors to make dense predictions of functionality in seen and unseen environments."



				  
			      

@InProceedings{Rhinehart2016CVPR,
  author = {Rhinehart, Nicholas and Kitani, Kris M.},
  title = {Learning Action Maps of Large Environments via First-Person Vision},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2016}
  } 

Visual Chunking: A List Prediction Framework for Region-Based Object Detection

N. Rhinehart, J. Zhou, M. Hebert, J. A. Bagnell

ICRA 2015 | pdf | show abs | show bib

Mini abstract: "We developed a principled imitation learning approach for the task of object detection, which is best described as a sequence prediction problem. Our approach reasons sequentially about objects, and requires no heuristics, such as Non-Maxima Suppression, to filter its predictions that are common in object detection frameworks."



				 
			      

@inproceedings{rhinehart2015visual,
  title={Visual chunking: A list prediction framework for region-based object detection},
  author={Rhinehart, Nicholas and Zhou, Jiaji and Hebert, Martial and Bagnell, J Andrew},
  booktitle={Robotics and Automation (ICRA), 2015 IEEE International Conference on},
  pages={5448--5454},
  year={2015},
  organization={IEEE}
}

Unrefereed Work


Flight Autonomy in Obstacle-Dense Environments

N. Rhinehart, D. Dey, J. A. Bagnell

Robotics Institute Summer Scholars Symposium, August 2011;
Sigma-Xi Research Symposium, October, 2011 poster (pdf) | youtube


Fast SFM-Based Localization of Temporal Sequences and Ground-Plane Hypothesis Consensus

Project for 16-822 Geometry Based Methods in Computer Vision, May, 2015

pdf | video (mp4)
Online Anomaly Detection in Video

Project for 16-831 Statistical Techniques in Robotics, December, 2014

pdf
Autonomous Localization and Navigation of Humanoid Robot

Swarthmore College Senior Thesis Project, May, 2012

pdf

Misc. undergrad projects


© Nick Rhinehart