Automatic Recognition of Food and Assessment of Diet

==========

This project is to apply the multimedia technology to the study of obesity. We are developing technologies that capture and process multimedia data that diet information. The ongoing research includes:

Multimedia Information Processing and Summarization

We are developing technologies to facilitate a human observer effectively analyzing these data. More specifically, we are developing algorithms and software modules for: 1) automatic dietary activity recognition from the chronically recorded video data, 2) data management and organization, and 3) user interfaces in the graphical, thumbnail, and storyboard forms for rapid data analysis in obesity research and clinical study. For example, an American, on average, spends one and a half hours on eating and drinking activities. We would like to able to detect these activities and summarize them into five minutes of the data. We will further provide analytic tools for the medical professionals for dietary assessment.

Privacy Protection in Video

In this project, the video camera is configured to record the same scene as the wearer’s perspective, which would raise privacy concerns. Although privacy protection in video is a poorly defined problem, we adopt the common measure in practice that protects the identity of people and content of objects from being recognized during the video playback. We are developing techniques to address privacy protection problems for both the subject who wears the device and other people. For example, the device may capture other people in the scene and patient’s own computer screen when he/she is using a computer. We will mask all human faces and computer screens in the captured video using object detection techniques.

Fast Food Database

Building a food database is a starting point for developing and testing food recognition programs for obesity study. Collaborating with Intel Pittsburgh research lab, we have built a fast food dataset, PFID (Pittsburgh Fast-food Image Dataset). The dataset contains a total of 4,545 still images, 606 stereo pairs, 303 3600 videos for structure from motion, and 27 privacy-preserving videos of eating events of volunteers. The dataset is public available at http://pfid.intel-research.net/

Related Publications:

M. Chen, K. Dhingra, W. Wu, L. Yang, R. Sukthankar, J. Yang, PFID:Pittsburgh Fast-food Image Dataset, Proceedings of International Conference on Image Processing (ICIP 2009).
W. Wu and J. Yang, Fast Food Recognition from Videos of Eating for Calorie Estimation, Proceedings of IEEE International Conference on Multimedia & Expo (ICME), 2009.
Q. Wang, J. Yang, Drinking Activity Analysis from Fast Food Eating Video Using Generative Models, ACM Int. Conf. on Multimedia (SIGMM) - Workshop on Cooking and Eating Activity, 2009
L. Yang, N. Zheng, H. Cheng, J. D. Fernstrom, M. Sun, J. Yang, Automatic Dietary Assessment from Fast Food Categorization, Proceedings of the IEEE 34th Annual Northeast Bioengineering Conference, 2008.
S. Greiner and J. Yang, Privacy Protection in an Electronic Chronicle System, Proceedings of the IEEE 34th Annual Northeast Bioengineering Conference, 2008.
L. Yang, J. Yang, N. Zheng, H. Cheng, Layered Object Categorization, Proceedings of 19th International Conference on Pattern Recognition, 2008.
D. Chen, Y. Chang, R. Yan, and J. Yang, Tools for Protecting the Privacy of Specific Individuals in Video, EURASIP Journal on Advances in Signal Processing, Vol. 2007, doi:10.1155/2007/75427, 9 pages, 2007.
N. Yao, R. J. Sclabassi, Q. Liu, J.Yang, J. D. Fernstrom, M. H. Fernstrom, and M. Sun, A Video Processing Approach to the Study of Obesity, Proceedings of 2007 IEEE International Conference on Multimedia and Expo (ICME 2007), pp.1019-1022, 2007.
Y. Chang, R. Yan, D. Chen, J. Yang, People Identification with Limited Labels in Privacy-Protected Video, Proceedings of IEEE International Conference on Multimedia & Expo 2006 (ICME 2006), pp. 1005-1008, 2006.

Back to the previous page

Last updated January, 2010