3D Human Kinematic Modeling and Markerless Motion Capture

German Cheung, Simon Baker, Takeo Kanade

Please contact German Cheung at german@ux2.sp.cs.cmu.edu for further details


Return to Homepage RealTime 3D Reconstruction Temporal SFS Human Kinematic Modeling and Markerless Motion Capture Human Motion Transfer

Objective

A vision-based human kinematic modeling system is built using the Temporal Shape-From-Silhouettes Algorithms (details can be found here ). The acquired kinematic model is then used to perform non-invasive motion capture of motions from video sequences using an idea similar to that used in our Temporal SFS Algorithms. Technical details of this project can be found in the documents listed below:


Human Kinematic Modeling System

There are three tasks to the human kinematic modeling system. The first task recovers all the joint locations of the body. The second task acquires precise shape information. The final task merges the joint and shape together to form a articulated model of the person.


Algorithms

Results

SubjectE SubjectG SubjectS
Subject
Joint
Estimation
Video
Video
Video
Joint
Skeleton
Shape
(Voxels)
Merged
Shape
and
Joints
Video
Video
Video



Human Markerless Motion Capture

The kinematic model acquired above is used to track new motions of the person by an Image-based Articulated Object Tracking Algorithm. Eight cameras are used in each sequence. Mpeg videos showing tracking results of two synthetic data sets (KICK and PUNCH sequences) and five real data sets of simple motions (STILLMARCH and AEROBICS sequences) and complex ones (KUNGFU, THROW, SLOWDANCE and STEP-FLEX sequences) are included below.

Synthetic Data Sets

KICK Sequence PUNCH Sequence


Real Data Sets

STILLMARCH Sequence
Video
AEROBICS Sequence
Video
KUNGFU Sequence
Video
THROW Sequence
Video
SLOWDANCE Sequence
Video
STEP-FLEX Sequence
Video



Applications for Human Motion Transfer

We also apply the above kinematic modeling and tracking results to an interesting application called Human Motion Transfer.