Manual methods of coding facial behavior are labor intensive, semi-quantitative, and difficult to standardize across laboratories or over time.

To capture the subtlety of human emotion and non-verbal communication, our interdisciplinary team of computer scientists and psychologists have been developing Automated Face Analysis System across its generations as shown below.

Generation
Condition
 Method
Target
Features Classifier
G1
 Frontal view
Limited head motion
Assumed planar motion 		Dense flow
Feature points
Edge power		 Hidden Markov Model
Discriminant Classifier
AUs occurring alone
(4 DOF head motion)

G2
Frontal view
Limited head motion
Assumed planar motion
Feature points
Gaussian mixtures
Edge power
Gabor coefficients 		Neural network
AUs occurring alone and in combinations
(4 DOF head motion)
G3
Target facial component is visible

 

  
AUs occurring in spontaneous behavior
(full 6 DOF head motion)


Continuing system development is part of a larger goal of developing computer systems that can detect human activity, recognize the people involved, understand their behavior, and respond appropriately.