Zoombie Dataset

(Hands under varying illumination)

Zoombie Mask

Zoombie Video

Raw C++ code
Simplified Version of CVPR 2013 paper (Per-Pixel Regression)
Labeling Tool (GrabCut Labeling Tool)

Updated Code from the CMU Art Fab:

Cheng Li and Kris M. Kitani.
Model Recommendation with Virtual Probes for Ego-Centric Hand Detection.
International Conference on Computer Vision (ICCV 2013). Dec 2013.

Cheng Li and Kris M. Kitani.
Pixel-level Hand Detection for Ego-centric Videos.
Conference on Computer Vision and Pattern Recognition (CVPR 2013). Jun 2013.

Activity Forecasting Dataset

Activity Forecasting Birds Eye View
Reward Function Soft Value Function

Optimal Control (OC) Demo:
Main function main.cpp
Optimal control class header oc.hpp
Optimal control class methods oc.cpp
Input files (image, feature maps, reward weights and terminal points) oc_demo.zip

Inverse Optimal Control (IOC) Demo:
Main function main.cpp
Inverse optimal control class header ioc.hpp
Inverse optimal control class methods ioc.cpp
Input files (basenames, images, features, trajectories, sample output) ioc_demo.zip

Observed tracker trajectories and ground truth trajectories tracker_output.zip

Rectified (top-down) images topdown_images.zip

Feature maps feature_maps.zip

Learned reward weights reward_weights.zip
Precomputed reward function for forecasting reward_function_forecasting.zip
Precomputed empirical feature counts empirical_feature_counts.zip
List of base names basenames.zip
General explanation of data content README.txt

Kris M. Kitani, Brian Ziebart, James A. Bagnell and Martial Hebert.
Activity Forecasting.
European Conference on Computer Vision (ECCV 2012), Oct 2012.

EgoAction Dataset

UEC Dataset (Choreographed Videos)

Quad Sequence
Quad sequence
QUAD.MP4.zip (254 MB)

Park Sequence
Park sequence
PARK.MP4.zip (1.56 GB)

YouTube Dataset (Non-choreographed Videos)

Mountain biking
Slope style
Horseback riding
Longboard Surfing

Ground Truth
Video segment labels

Class for online inference with Dirichlet Process Mixture Models onlineDP.zip
Class wrapper for the OpenCV sparse optical flow optflow.zip
Class for computing the motion histogram given a set of point correspondences mohist.zip

Citation Kris M. Kitani, Takahiro Okabe, Yoichi Sato, and Akihiro Sugimoto.
Fast Unsupervised Ego-Action Learning for First-person Sports Videos.
Conference on Computer Vision and Pattern Recognition (CVPR2011), June 2011.


  1. Do results depend on my image size?
    Yes, the flow magnitude and flow variance bin thresholds are optimized for WVGA (840 x 480)

  2. Does optflow require any libraries?
    Yes, you need to have OpenCV. The code should work with version 2.0 and above.

  3. Do I need to calibrate for lens distortion?
    No, parameters are actually optimized for the GoPro lens with radial distortion.