Nick Rhinehart


Ph.D. student @ Robotics Institute, Carnegie Mellon University


Contact & Info

RI Profile

About Me

Hi, I'm Nick Rhinehart, a Ph.D student at the CMU Robotics Institute. I work in machine learning and computer vision with Kris Kitani and Drew Bagnell. My research broadly investigates automatic computer vision-driven human-centric understanding.

I'm currently focused on:
  • building human-centric representations of the world
  • predicting a person's long-term intentions
  • learning to help humans achieve their goals
  • Research interests include: human activity analysis, activity forecasting, online learning, reinforcement & imitation learning, first-person vision, and artificial intelligence

    Recent News

  • Dec 2016: New pre-print, DARKO, on arXiv
  • March 2016: New paper, Action Maps, at CVPR 2016
  • Summer 2016: R&D at the Uber Advanced Technologies Center
  • Summer 2015: I visited & collaborated with the Sato Laboratory at the University of Tokyo
  • May 2015: New paper, Visual Chunking, at ICRA 2015
  • Fall 2014: NIPS 2014 workshop presentation
  • Pre-prints

    Online Semantic Activity Forecasting with DARKO

    N. Rhinehart, K. Kitani
    [.pdf] • [arXiv] • [Project page] • [Video]
    We address the problem of continuously observing and forecasting long-term semantic activities of a first-person camera wearer: what the person will do, where they will go, and what goal they are seeking. In contrast to prior work in trajectory forecasting and short-term activity forecasting, our algorithm, DARKO, reasons about the future position, future semantic state, and future high-level goals of the camera-wearer at arbitrary spatial and temporal horizons defined only by the wearer's behaviors. DARKO learns and forecasts online from first-person observations of the user's daily behaviors. We derive novel mathematical results that enable efficient forecasting of different semantic quantities of interest. We apply our method to a dataset of 5 large-scale environments with 3 different environment types, collected from 3 different users, and experimentally validate DARKO on forecasting tasks.

    Refereed Research

    Learning Action Maps of Large Environments Via First-Person Vision

    N. Rhinehart, K. Kitani
    CVPR, 2016; MACV 2016
    [.pdf] • [arXiv] • [Bibtex] • [YouTube] • [Slides (.pdf)]
    Our goal is to automate dense functional understanding of large spaces by leveraging sparse activity demonstrations recorded from an ego-centric viewpoint. We apply regularized nonnegative matrix factorization to estimate functionality in large scenes where people have behaved, as well as novel scenes where no behaviors are observed. Our method learns and predicts "Action Maps", which encode the ability for a user to perform activities at various locations.

    Visual Chunking: A List Prediction Framework for Region-Based Object Detection

    N. Rhinehart, J. Zhou, M. Hebert, J. A. Bagnell
    ICRA, 2015
    [.pdf] • [.bib] • [Poster (.key)] • [Poster (.ppsx)] • [Poster (.pdf)] • [Youtube]
    We detect objects in images by learning to select from a set of candidate regions. Our generic approach, which we term "Visual Chunking", reasons about the locations of multiple object instances in an image while expressively describing object boundaries. We design an optimization criterion for measuring the performance of a list of such detections as a natural extension to a common per-instance metric, and apply imitation-learning methods to sequentially predict objects in new images.

    Fine-Grained Detection via Efficient Extreme Classification

    N. Rhinehart, J. Zhou, M. Hebert, J. A. Bagnell
    NIPS 2014, Presentation at Workshop on Extreme Classification.
    [Poster (.pdf) ] • [Poster (.pptx)]

    Unrefereed Research

    Flight Autonomy in Obstacle-Dense Environments

    N. Rhinehart, D. Dey, J. A. Bagnell
    Robotics Institute Summer Scholars Symposium, August 2011;
    Sigma-Xi Research Symposium, October, 2011
    [Poster (.pdf)] • [Youtube]

    Autonomous Localization and Navigation of Humanoid Robot

    N. Rhinehart, M. Zucker
    Swarthmore College Senior Thesis Project, May, 2012

    Other Unrefereed Work

    Fast SFM-Based Localization of Temporal Sequences and Ground-Plane Hypothesis Consensus

    Project for 16-822 Geometry Based Methods in Computer Vision, May, 2015
    [.pdf] • [Video (.mp4)]

    Online Anomaly Detection in Video

    Project for 16-831 Statistical Techniques in Robotics, December, 2014
    Miscellaneous Projects Projects
    © 2013-2016
    Nick Rhinehart