Junwei Liang

 
			Junwei Liang
			(Ph.D. Candidate)      
			梁俊卫
		
			Language Technologies Institute
		
			School of Computer Science
		
			Carnegie Mellon University
		
			Office: Gates and Hillman Centers 5513
		
			Email: junweil at cs dot cmu dot edu (junweil@cs.cmu.edu)

Publications

Research

Honors

Google Scholar

Code

Introduction

Greetings! I'm a 2nd year Ph.D. student under the supervision of Prof. Alexander Hauptmann at Carnegie Mellon University.
My research interests are large-scale computer vision and natural language processing.
I am also doing research/building tools for event reconstruction from unorganized social media videos for public safety projects.

News

[04/2019] Our CMU team's (INF & MUDSML) system achieved best performance on the activity detection challenge in surveillance videos hosted by NIST & IARPA. We have released our code and model for Object Detection & Tracking here.
[02/2019] Future activity & path prediction paper reported by the media with 30k+ views in a week. To be presented at CVPR 2019.
[12/2018] MemexQA paper accepted by TPAMI (IEEE Transactions on Pattern Analysis and Machine Intelligence).
[11/2018] Runner-up in the NIST TRECVID 2018 Ad-hoc Video Search Challenge.
[06/2018] Presented MemexQA paper at CVPR 2018. [Presentation]
[02/2017] Presented the Event Reconstruction Tool demo at AAAI 2017. [demo]

Research Experience

Research Assistant at Carnegie Mellon University
2015 - present

Worked on Large-scale Video Analysis and Retrieval. Studied unsupervised learning of video concept detectors from the Internet. Also participated in the development of event reconstruction tool. The project is for Synchronization and localization of noisy user-generated videos to reconstruct the event scene and timeline from unorganized social media videos, affiliates with CMU Center for Human Rights Science. Advised by Prof. Alexander Hauptmann.
Research Intern at Google AI
May 2019 - Aug 2019

Working on changing the world.
Student Researcher at Google Cloud AI
May 2018 - Dec 2018

Worked on activity recognition and prediction in multi-perspective streaming videos. Studied principal computer vision and high-level semantic reasoning models for interperson and person-object interaction to help AI better understand human activities. Advised by Lu Jiang.
Research Assistant at Renmin University of China
2013 - 2015

Studied semantic concept annotation on user-generated videos using audio. Participated HUAWEI semantic concept annotation of UGC videos grand challenge 2014 and ranked 3rd in the evaluation. Also worked on natural language description generation for images and videos with deep models. Ranked 1st in ImageCLEF 2015 “image to sentence” subtask in the evaluation.

Honors and Awards

CVPR, IJCAI, NIST PSCR, NIST TRECVID, Inmind student travel grants
2016-2019
Yahoo! Fellowship (Stipend and tuition in CMU)
2016-2018
Best Undergraduate Thesis (Top 5%)
2015
Second Prize, the National Undergraduates Computer Design Competition of China
2014
National Prize (Top 10%), National Undergraduates Innovation Project
2013

Selected Publications [Google Scholar]

Peeking into the Future: Predicting Future Person Activities and Locations in Videos
Junwei Liang, Lu Jiang, Juan Carlos Niebles, Alexander Hauptmann, Li Fei-Fei

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. (Translated and reported by a Chinese media (量子位, 02/13/2019), with 30k+ views in a week.)

Paper BibTex Demo Video Project Page/Code/Model
Focal Visual-Text Attention for Memex Question Answering
Junwei Liang, Lu Jiang, Liangliang Cao, Yannis Kalantidis, Li-Jia Li, and Alexander Hauptmann

In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). (Journal Regular Paper)

Paper BibTex Code/Model/Dataset
Focal Visual-Text Attention for Visual Question Answering
Junwei Liang, Lu Jiang, Liangliang Cao, Li-Jia Li, and Alexander Hauptmann

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. (Spotlight Paper, 6.8% acceptance rate)

Paper BibTex Code/Model
Leveraging Multi-modal Prior Knowledge for Large-scale Concept Learning in Noisy Web Data
Junwei Liang, Lu Jiang, Deyu Meng, and Alexander Hauptmann

In ACM International Conference on Multimedia Retrieval (ICMR), 2017.

Paper BibTex
Webly-Supervised Learning of Multimodal Video Detectors
Junwei Liang, Lu Jiang, and Alexander Hauptmann

In AAAI Conference on Artificial Intelligence (AAAI), 2017.

Paper BibTex demo Poster
Learning to detect concepts from webly-labeled video data
Junwei Liang, Lu Jiang, Deyu Meng, and Alexander Hauptmann

In International Joint Conference on Artificial Intelligence (IJCAI), 2016.

Paper BibTex
Video Description Generation using Audio and Visual Cues
Qin Jin, and Junwei Liang

In ACM International Conference on Multimedia Retrieval (ICMR), 2016.

Paper BibTex
Temporal Localization of Audio Events for Conflict Monitoring in Social Media
Junwei Liang, Lu Jiang and Alexander Hauptmann

In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2017.

Paper BibTex poster

Educations

Ph.D. in Language and Information Technology
2017 - present

School of Computer Science, Carnegie Mellon University

Advisor: Alexander Hauptmann
M.S. in Language Technologies
2015 - 2017

School of Computer Science, Carnegie Mellon University

Advisor: Alexander Hauptmann
B.S. in Computer Science
2011 - 2015

School of Information, Renmin University of China

Advisor: Qin Jin

Selected Professional Activities

Journal Reviewer
ACM TOMM 2018

Neurocomputing 2019

IEEE Transactions on Multimedia 2019

IEEE Access 2019
Conference Program Committee
ACM Multimedia 2017

ACM Multimedia 2019
Website Developer (2014-2015)
Designed and developed major CMS websites for organizations. Hillhouse Academy, Annual Computer Design Competition in China (>2000 users annually)