Human Sensing

Enabling computers to understand and characterize human behaviour has the potential to revolutionize many areas that benefit society such as clinical diagnosis, human computer interaction, and social robotics. In the Human Sensing (HS) laboratory we are interested in modelling and characterizing human behaviour from a variety of sensory data (e.g., video, motion capture, wearable sensors). Some projects include:

  • 1- Activity and event detection from video
  • 2- Depression detection from multimodal data (e.g., RGBD video, binary sensors and accelerometers)
  • 3- Monitoring Parkinson's desease with wearable sensors
  • 4- Automatic stress analysis from psychological measurements (e.g., EEG, GSR, heart rate)
  • 5- Hot-flash detection with wearable GSR sensors
  • 6- Personalization of Machine Learning algorithms


Augmented Reality
Virtual Reality

Augmented Reality(AR) and Virtual Reality (VR) are likely to be the fourth transformational technology in the area of human computing (after the introduction of personal computing, www, and the iPhone). Our digital lives will no longer be in a smart phone, but AR glasses and VR headsets that will allow to superimpose the real world with the digital one. Our goal is to develop algorithms for human sensing where the computer generated image will be mixed so tightly with reality, that we will not be able to tell what is real from what is not. We believe that AR/VR will provide the next platform generation for human computing. Some current projects of interest are:

  • 1- Applications of AR Glasses
  • 2- Facial expression transfer from VR headsets
  • 3- Virtual avatars
  • 4- Neural rendering of bodies and faces
  • 5- Wifi-based human detection


Data Focused CV and
Generative Models

Having access to large amounts of well-balanced, and well-labeled visual data is arguably the most important component of any learning-based computer vision system. On the other hand, labeling and collecting privacy-preserving data is typically the most time-consuming, expensive, and error-prone step in building these systems. However, most research in computer vision has been focused on improving the speed or accuracy of neural models. Our goal is to work on computer vision algorithms to improve the speed, precision, and creation of balanced datasets. We use generative models for:

  • 1- Synthetic data generation (close the domain gap)
  • 2- Detect where models fail
  • 3- Automatic detection of bias in datasets
  • 4- Active Learning
  • 5- Generative models for data augmentation
  • 6- Automatic rebalancing of datasets