15-869 Human Motion Modeling and Analysis

Instructors: Yaser Sheikh (CMU), Leonid Sigal (Disney Research), Iain Matthews (Disney Research)
MW 3:00-4:20 GHC 5222

Office Hours
Yaser Sheikh: EDSH 221 Tuesday 2pm-3pm
Leon Sigal: CIC (Lower Level) Thursday 1pm-2pm

Human motion analysis is used in applications as varied as special effects in movies, animation, sport training, physical rehabilitation for the disabled, and human-robot/human-computer interaction. This course will survey state-of-the-art techniques, in the industry and academia, to capture, model, and analyze human motion. The course will be a mix between lectures and seminar-style paper reading of recent research into human motion modeling and analysis. The course evaluation will be project-based, in which students will capture their own body and face motion, and build projects around the data they collect individually and as a group.

In particular, we will cover:
Capture Techniques: We will describe and use various systems including motion capture, video-based capture, depth sensors, scanners, and eye-gaze trackers.
Modeling and Representation: We will cover classic and contemporary representations of face and body pose and motion, including statistical and physics-based techniques.
Applications: As human motion analysis becomes increasingly mature, new applications are emerging. We will study recent progress in animation, synthesis, classification, and rehabilitation.

Grading Criteria


  Date Topic Material
Lecture 1 September 10 Introduction to Human Motion [ PDF ]
Part I: Capture
Lecture 2 September 12 Virtualizing Reality: Introduction to markerless motion capture
Reading Assignment: T. Kanade and P. J. Narayanan, Virtualized Reality: Perspectives on 4D Digitization of Dynamic Events, IEEE Computer Graphics and Applications, 2007.
[ PDF ]
Lecture 3 September 17 Marker-based Capture: Introduction to marker-based motion capture
Reading Assignment: G. Welch and E. Foxlin, Motion tracking: No silver bullet, but a respectable arsenal, IEEE Computer Graphics and Applications 2002.
[ PDF ]
Lecture 4 September 19 Motion Capture Lab Demo: Live demonstration of motion capture at the CMU motion capture lab.  
Lecture 5 September 24 Game Capture: Introduction to game capture tech.
Reading Assignment: Jamie Shotton et al., Real-Time Human Pose Recognition in Parts from a Single Depth Image, CVPR 2011.
[ PDF ]
Lecture 6 September 26 Facial Capture: Introduction to face capture tech.
Reading Assignment: T. Beeler et al., High-quality passive facial performance capture using anchor frames, ACM Transactions on Graphics, 2011.
[ PDF ]
Part II: Pose Modeling
Lecture 7 October 1 Human Body Representation: Articulated Systems, Anatomical Classification
Reading Assignment: J. O'Brien, R. Bodenheimer, G. Brostow, and J. Hodgins Automatic Joint Parameter Estimation from Magnetic Motion Capture Data, 2000.
[ PDF ]
Lecture 8 October 3 Facial Representations: Facial Action Units, Blendshapes, Pose Space Deformations
Reading Assignment: J. P. Lewis, Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation, SIGGRAPH, 2000.
[ PDF ]
Lecture 9 October 8 Mesh Representation: Triangulation, Skinning
Reading Assignment: B. Allen et al., The space of all body shapes: reconstruction and parameterization from range scans. SIGGRAPH 2003.
[ PDF ]
Lecture 10 October 10

Project Pitch: Students pitch their final projects for comments and discussion. Each student will have 3 minutes to present

Lecture 11 October 15

Capture Project Review: Student teams will present their capture results. Each team will have 15 minutes to present
Group 1: Motion Capture
Group 2: Face Scans
Group 3: Multiview Performance
Group 4: Face Motion Capture

Part III: Motion Modeling & Applications
Lecture 12 October 17 Statistical Models I: Latent Variable Models, PCA, GPLVMs, Isomap
Reading Assignment: Safanova et al., Synthesizing Physically Realistic Human Motion in Low-Dimensional, Behavior-Specific Spaces.
[ PDF ]
Lecture 13 October 22 Application: Motion Retargeting
Paper 1: Semantic Deformation Transfer
Paper 2: Style-based Inverse Kinematics
Lecture 14 October 24 Statistical Models II: Linear Dynamical Models
Reading Assignment: D. Fleet, Motion models for people tracking. Guide to Visual Analysis of Humans: Looking at People, 2011.
[ PDF ]
Lecture 15 October 29 Application: Animation
Paper 1: Motion Graphs
Paper 2: GPDMs
Lecture 16 October 31 Physical Models I: Spacetime Constraints
Reading Assignment: Witkin and Kass. Spacetime Constraints, Computer Graphics, 1988.
Lecture 17 November 5 Application: Motion Control
Paper 1: Optimization-based Interactive Motion Synthesis
Paper 2: Control Systems for Human Running using an Inverted Pendulum Model and a Reference Motion Capture Sequence
Lecture 18 November 7 Statistical Models III: Bilinear Spatiotemporal Models
Reading Assignment: Akhter et al., Bilinear Spatiotemopral Basis Models, ACM Transactions on Graphics, 2012.
Lecture 19 November 12 Final Project Discussion  
Lecture 20 November 14 Application: Nonrigid Structure from Motion
Paper 1: Estimating Shape and Motion with Hierarchical Priors
Paper 2: Face Transfer with Multilinear Models
Lecture 21 November 19 Application: Locomotion Controllers
Paper 1: Optimizing Locomotion Controllers Using Biologically-Based Actuators and Objectives
Paper 2: SIMBICON: Simple Biped Locomotion Control
Lecture 22 November 26

Physical Models II: Controllers
Reading Assignment: Hodgins et al., Animating human athletics, SIGGRAPH 1995.

Part IV: Miscellaneous
Lecture 23 December 3 Techniques in Human Action Recognition  
Lecture 24 December 5 Human Motion Perception  
Lecture 25 December 12 Project Presentations