Katerina Fragkiadaki Home

Katerina Fragkiadaki

I am interested in building machines that understand the stories that videos portray, and, inversely, in using videos to teach machines about the world. The pen-ultimate goal is to build a machine that understands movie plots, and the ultimate goal is to build a machine that would want to watch Bergman over this.

I am an Assistant Professor in the Machine Learning Department at Carnegie Mellon. Prior to joining MLD's faculty I spent three wonderful years as a post doctoral researcher first at UC Berkeley working with Jitendra Malik and then at Google Research in Mountain View working with the video group. I completed my Ph.D. in GRASP, UPenn with Jianbo Shi. I did my undergraduate studies at the National Technical University of Athens and before that I was in Crete.

Dear prospective graduate students: There is no need to email, please submit your application to our School following the links here: MLD Ph.D. or R.I. Ph.D.

8107 Gates Hillman
Carnegie Mellon University
Pittsburgh, PA, 15213
katef 'at' cs.cmu.edu
Admin: Amy Protos, aprotos@cs.cmu.edu

News

I gave a talk on Adversarial Inverse Graphics Networks at Augmenting humans workshop, CVPR 2019
I gave a talk on Embodied Visual recognition at Bringing Robots to the Computer Vision Community Workshop, CVPR, 2019
I gave a talk on Embodied language Grounding at The How2 Challenge: New Tasks for Vision and Language Workshop, ICML 2019
I gave a talk on geometry-aware deep visual learning at the theory and practice in ML and CV workshop at ICERM, slides and video are available.
I was awarded a UPMC faculty research award 2019
Ricson is selected as a runner-up of the CRA Outstanding Undergraduate Researcher Award for 2019 !
Area chair ICML 2019
Area chair ICLR 2019
We are organizing an AI4ALL summer school to expose rising juniors to AI and the good it can do for the world, as we see it, here in CMU!
I gave a talk at the Deep Learning for Visual SLAM workshop in CVPR 2018, and the slides are available here.
Area chair CVPR 2018
I got a Google faculty award 2018

Teaching

Deep Reinforcement Learning and Control Spring 2017
Language Grounding for Vision and Control Fall 2017
Deep Reinforcement Learning and Control Fall 2018
Deep Reinforcement Learning and Control Spring 2019

Research

Teaching machines to appreciate Bergman is a hard problem, in fact, most people fail in it. Thus, it is a very long term goal of our research. Meanwhile, we are working on teaching machines basic common sense. Our recent works use recurrent nets with 3D representation bottlenecks that disentangle camera motion from appearance, while being optimized end-to-end for a final task, such as view prediction or 3D object detection. In this way, our models learn object permanence, size constancy, occlusions and dis-occlusion relationships, useful for 3D object detection, affordance learning and language grounding.

Selected Publications

	Embodied View-Contrastive 3D Feature Learning Adam W. Harley, Fangyu Li, Shrinidhi K. Lakshmikanth, Xian Zhou, Hsiao-Yu Fish Tung, Katerina Fragkiadaki arxiv paper
	Learning Spatial Common Sense with Geometry-Aware Recurrent Networks Hsiao-Yu Fish Tung, Ricson Cheng, Katerina Fragkiadaki CVPR 2019, oral presentation paper \| project page with code
	Model Learning for Look-ahead Exploration in Continuous Control Arpit Agarwal, Katharina Muelling and Katerina Fragkiadaki AAAI 2019, oral presentation paper \| project page with code
	Data Dreaming for Object Detection: Learning Object-Centric State Representations for Visual Imitation Maximilian Sieb and Katerina Fragkiadaki Humanoids 2018, oral presentation paper \| slides
	Reinforcement Learning of Active Vision for Manipulating Objects under Occlusionss Ricson Cheng, Arpit Agarwal, and Katerina Fragkiadaki CoRL 2018 paper \| slides \| code
	Geometry-Aware Recurrent Neural Networks for Active Visual Recognition Ricson Cheng, Ziyan Wang, and Katerina Fragkiadaki NIPS 2018 paper
	Reward Learning from Narrated Demonstrations Fish Tung, Adam Harley, Liang-Kang Huang, Katerina Fragkiadaki CVPR 2018 paper \| bibtex
	Depth-adaptive Computational Policies for Efficient Visual Tracking Chris Ying, Katerina Fragkiadaki EMMCVPR 2017 paper \| bibtex
	Self-supervised Learning of Motion Capture Hsiao-Yu Fish Tung, Wei Tung, Ersin Yumer, Katerina Fragkiadaki NIPS 2017, spotlight paper \| bibtex \| code
	Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and mage-to-Image Translation from Unpaired Supervision Hsiao-Yu Fish Tung, Adam Harley, William Seto and Katerina Fragkiadaki ICCV 2017 paper \| bibtex \| code
	SfM-Net: Learning of Structure and Motion from Video Sudheendra Vijayanarasimhan, Susanna Ricco, Cordelia Schmid, Rahul Sukthankar and Katerina Fragkiadaki arxiv
	Motion Prediction Under Multimodality with Conditional Stochastic Networks Katerina Fragkiadaki, Jonathan Huang, Alex Alemi, Sudheendra Vijayanarasimhan, Susanna Ricco and Rahul Sukthankar arxiv \| video results
	Learning Feature Hierarchies from Long-Range Temporal Associations in Videos Panna Felsen, Katerina Fragkiadaki, Jitendra Malik and Alexei Efros Workshop on Transfer and Multi-task Learning, in conjunction with NIPS 2015 paper
	Learning Predictive Visual Models of Physics for Playing Billiards Katerina Fragkiadaki, Pulkit Agrawal, Sergey Levine and Jitendra Malik ICLR 2016 paper \| project page
	Recurrent Network Models for Human Dynamics Katerina Fragkiadaki, Sergey Levine, Panna Felsen and Jitendra Malik ICCV 2015 paper \| project page
	Human Pose Estimation with Iterative Error Feedback Joao Carreira, Pulkit Agrawal, Katerina Fragkiadaki, and Jitendra Malik arXiv paper\|project page with code
	Learning to Segment Moving Objects in Videos Katerina Fragkiadaki, Pablo Arbelaez, Panna Felsen and Jitendra Malik CVPR 2015 paper \| poster \| bibtex \| project page with code
	Grouping-Based Low-Rank Video Completion and 3D Reconstruction Katerina Fragkiadaki, Marta Salas, Pablo Arbelaez, and Jitendra Malik NIPS 2014 paper \| poster \| bibtex \| project page with code
	Two Granularity Tracking: Mediating Trajectory and Detection Graphs for Tracking under Occlusions Katerina Fragkiadaki, Weiyu Zhang, Geng Zhang, and Jianbo Shi ECCV 2012 paper \| poster \| bibtex \| project page with code
	Video Segmentation by tracing Discontinuities in a Trajectory Embedding Katerina Fragkiadaki, Geng Zhang, and Jianbo Shi CVPR 2012 paper \| poster \| bibtex \| project page with code
	Detection-free Tracking: Exploiting Motion and Topology for Segmenting and Tracking under Entanglement Katerina Fragkiadaki and Jianbo Shi CVPR 2011 paper \| poster \| bibtex

Current Graduate Students

Adam Harley, R.I. Ph.D.

Fish Tung, MLD Ph.D.

Xian Zhou, R.I. Ph.D.

Ricson Chen, undergrad CSD

Ziyan Wang, R.I. Masters

Max Sieb, R.I. Masters (co-advised with Oliver Kroemer)

Alumni

Arpit Agarwal, R.I. Masters

Henry Huang, MLD Masters

Chris Ying, MLD Masters

Yijie Wang, MLD Masters

MISC

Link to the 10th Perceptual Organization for Computer Vision Workshop in conjunction with CVPR 2016 with a focus on the role of feedback and sequential processing for recognition and motion perception
Link to the 2nd International Workshop on Video Segmentation in conjunction with ECCV 2016 with a focus the role of segmentation for driving learning