Nick Rhinehart


Ph.D. student @ Robotics Institute, Carnegie Mellon University


Contact & Info

RI Profile

About Me

Hi, I'm Nick Rhinehart, a Ph.D student at the CMU Robotics Institute. I'm interested in Machine-Learning driven Computer Vision in activity recognition, activity forecasting, and functional understanding. I work with Kris Kitani and Drew Bagnell.

Research interests include: human activity analysis, activity forecasting, online learning, reinforcement & imitation learning, first-person vision, and artificial intelligence

Recent News

  • Dec 2016: New pre-print, DARKO, on arXiv
  • March 2016: New paper, Action Maps, at CVPR 2016
  • Summer 2016: R&D at the Uber Advanced Technologies Center
  • Summer 2015: I visited & collaborated with the Sato Laboratory at the University of Tokyo
  • May 2015: New paper, Visual Chunking, at ICRA 2015
  • Fall 2014: NIPS 2014 workshop presentation
  • Pre-prints

    First-Person Forecasting with
    Online Inverse Reinforcement Learning

    N. Rhinehart, K. Kitani
    [.pdf] • [arXiv] • [Project page] • [Video]
    We address the problem of incrementally modeling and forecasting long-term goals of a first-person camera wearer: what the user will do, where they will go, and what goal they are attempting to reach. In contrast to prior work in trajectory forecasting, our algorithm, DARKO, goes further to reason about semantic states (will I pick up an object?), and future goal states that are far both in terms of space and time. DARKO learns and forecasts from first-person visual observations of the user's daily behaviors via an Online Inverse Reinforcement Learning (IRL) approach. Classical IRL discovers only the rewards in a batch setting, whereas DARKO discovers the states, transitions, rewards, and goals of a user from streaming data. Among other results, we show DARKO forecasts goals better than competing methods in both noisy and ideal settings, and our approach is theoretically and empirically no-regret.

    Refereed Research

    Learning Action Maps of Large Environments Via First-Person Vision

    N. Rhinehart, K. Kitani
    CVPR, 2016; MACV 2016
    [.pdf] • [Slides (.pdf)] • [Bibtex] • [arXiv]
    Our goal is to automate dense functional understanding of large spaces by leveraging sparse activity demonstrations recorded from an ego-centric viewpoint. We apply regularized nonnegative matrix factorization to estimate functionality in large scenes where people have behaved, as well as novel scenes where no behaviors are observed. Our method learns and predicts "Action Maps", which encode the ability for a user to perform activities at various locations.

    Visual Chunking: A List Prediction Framework for Region-Based Object Detection

    N. Rhinehart, J. Zhou, M. Hebert, J. A. Bagnell
    ICRA, 2015
    [.pdf] • [.bib] • [Poster (.key)] • [Poster (.ppsx)] • [Poster (.pdf)] • [Youtube]
    We detect objects in images by learning to select from a set of candidate regions. Our generic approach, which we term "Visual Chunking", reasons about the locations of multiple object instances in an image while expressively describing object boundaries. We design an optimization criterion for measuring the performance of a list of such detections as a natural extension to a common per-instance metric, and apply imitation-learning methods to sequentially predict objects in new images.

    Fine-Grained Detection via Efficient Extreme Classification

    N. Rhinehart, J. Zhou, M. Hebert, J. A. Bagnell
    NIPS 2014, Presentation at Workshop on Extreme Classification.
    [Poster (.pdf) ] • [Poster (.pptx)]

    Unrefereed Research

    Flight Autonomy in Obstacle-Dense Environments

    N. Rhinehart, D. Dey, J. A. Bagnell
    Robotics Institute Summer Scholars Symposium, August 2011;
    Sigma-Xi Research Symposium, October, 2011
    [Poster (.pdf)] • [Youtube]

    Autonomous Localization and Navigation of Humanoid Robot

    N. Rhinehart, M. Zucker
    Swarthmore College Senior Thesis Project, May, 2012

    Other Unrefereed Work

    Fast SFM-Based Localization of Temporal Sequences and Ground-Plane Hypothesis Consensus

    Project for 16-822 Geometry Based Methods in Computer Vision, May, 2015
    [.pdf] • [Video (.mp4)]

    Online Anomaly Detection in Video

    Project for 16-831 Statistical Techniques in Robotics, December, 2014
    Miscellaneous Projects Projects
    © 2013-2016
    Nick Rhinehart