Predicting Primary Gaze Behavior
using Social Saliency Fields

Hyun Soo Park¹, Eakta Jain², and Yaser Sheikh¹

¹Carnegie Mellon University
²Texas Instruments

Figure 1: We model the relationship between a primary gaze direction and a social charge via a social saliency field inspired by Coulomb's law. The two social charges (the blue and green points) generate the social saliency saliency field on the left figure. The size of the social charges is proportional to social saliency. In the right figure, we show the probability distribution over gaze direction modeled by a mixture of von-Mises Fisher distributions.

Figure 2: We estimate a social saliency field that reflects the selective gaze behavior. An arrow represents the predicted primary gaze direction at the location.

Abstract

We present a method to predict primary gaze behavior in a social scene. Inspired by the study of electric fields, we posit ``social charges" - latent quantities that drive the primary gaze behavior of members of a social group. These charges induce a gradient field that defines the relationship between the social charges and the primary gaze direction of members in the scene. This field model is used to predict primary gaze behavior at any location or time in the scene. We present an algorithm to estimate the time-varying behavior of these charges from the primary gaze behavior of measured observers in the scene. We validate the model by evaluating its predictive precision via cross-validation in a variety of social scenes.

Paper

Hyun Soo Park, Eakta Jain, and Yaser Sheikh "Predicting Primary Gaze Behavior using Social Saliency Fields" International Conference on Computer Vision (ICCV), Dec., 2013, [PDF, bib]

Video

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grants No. 1353120 and 1029679. In addition, this research was supported by the Intel ISTC on Embedded Computing. We also thank Zijun Wei for the facial detection code and Natasha Kholgade for comments and suggestions on this work.