Gus XIA

My Research: Towards a more creative, expressive, interactive world via Music AI.

I am broadly interested in the design of interactive intelligent systems to extend human musical creation and expression. This research lies in the intersection of Machine Learning, HCI, Robotics, and Computer Music. My representative works include interactive composition via style transfer, human-computer interactive performances, autonomous dancing robots, large-scale content-based music retrieval, haptic guidance for flute tutoring, and bio-music computing using slime mold.

Particularly, I build music agents that compose and arrange music via style transfer and analogy, perform and improvise music expressively in concert with human musicians by learning from rehearsal experience, and tutor music beginners using haptic guidance. These efforts unify expressive performance rendering, automatic accompaniment, and algorithmic composition in a machine-learning framework, making music a more accessible and friendly tool for everyone.

About me: A computer scientist and professional musician.

I am an Assistant Professor in Computer Science at NYU Shanghai. I received my Ph.D. in the Machine Learning Department at Carnegie Mellon University where I study Machine Learning and Computer Music under the advice of Prof. Roger Dannenberg. I was a Neukom Fellow at Dartmouth from 2016 to 2017. In 2010, I received my undergraduate degree in Information Science with a minor in Psychology at Peking University. I am also a professional DI and XIAO (Chinese flute and vertical flute) player. I was the prime soloist of the Chinese Music Institute (CMI) of Peking University, where I also served as the president and assistant conductor. I held my solo concert in 2010. (See Music Events at the bottom.)


News:
I am looking for passionate PhD and undergraduate students to join my Music X Lab at NYU Shanghai.

My 2020 Spring courses on Computer Music and Machine Learning will be online.
Representative Publications

Deep Music Generation:

Authors Title Year Book/Journal/Proceedings
S. Dai and G. Xia. Music Style Transfer: A Position Paper[PDF] 2018 6th International Workshop on Musical Metacreation
R. Yang, D. Wang, Z. Wang, T. Chen, J. Jiang and G. Xia. Deep Music Analogy Via Latent Representation Disentanglement [PDF] 2019 Proc. 20th the International Conference on Music Information Retrieval (ISMIR)
J. Jiang, G. Xia, and R. Dannenberg Representing Music Structure by Variational Attention
[PDF]
2019 ML4MD workshop at ICML
J. Jiang and G. Xia. Transformer VAE: A Hierarchical Model for Structure-aware and Interpretable Music Representation Learning
[PDF]
2020 Proc. 45th International Conference on Acoustics, Speech and Signal Processing

Computer-aided Music Learning via Multimodal feedbacks

Authors Title Year Book/Journal/Proceedings
G. Xia, C. Jacobsen, Q. Chen, X-D. Yang, and R. Dannenberg ShIFT: A Semi-haptic Interface for Flute Tutoring
[PDF]
2018 Proc. 18th The International Conference on New Interfaces for Musical Expression (NIME)
Y. Zhang, Y. Li, D. Chin, and G. Xia Adaptive Multimodal Music Learning via Interactive-haptic Instrument
[PDF]
2019 Proc. 19th The International Conference on New Interfaces for Musical Expression (NIME)

Expressive and Interactive Performance

Authors Title Year Book/Journal/Proceedings
G. Xia & R. Dannenberg Duet Interaction: Learning Musicianship for Automatic Accompaniment.
[PDF] [Slides] [Audio Demo] [Data]
2015 Proc. 15th the International Conference on New Interfaces for Musical Expression (NIME)
G. Xia, K. Mao, M. Fu, S. Cosentino, G.Trovato, S. Sessa, A. Takanishi, and R. Dannenberg Expressive Humanoid Robot for Automatic Accompaniment.
[PDF] [Slides]
2016 Proc. 13th the International Conference on Sound and Music Computing (SMC)
G. Xia, J. Tay, R. Dannenberg, and M. Veloso Autonomous Robot Dancing Driven by Beats and Emotions of Music.
[PDF] [Slides] [Demo1] [Demo2]
2012 Proc. 12th International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS)
G. Xia, Y. Wang, R. Dannenberg, and G. Gordon Spectral Learning for Expressive Interactive Ensemble Music Performance.
[PDF] [Slides] [Audio Demo]
2015 Proc. 16th the International Conference on Music Information Retrieval (ISMIR)
S. Dai, and G. Xia, Computational Models for Common Pipa Techniques [PDF coming soon] 2017 Proc. 5th  National Conference on Sound and Music Technology (CSMT) Best student paper award
G. Xia, and R. Dannenberg Duet Interaction: Learning Improvisation Techniques for Automatic Accompaniment
[PDF]
2017 Proc. 17th The International Conference on New Interfaces for Musical Expression (NIME)
D. Liang, G. Xia, and R. Dannenberg A Framework for Coordination and Synchronization of Media.
[PDF] [Slides] [Talk + Demo]
2011 Proc. 11th the International Conference on New Interfaces for Musical Expression (NIME)
M. Xu, Z. Wang, and G. Xia Transferring Piano Performance Control Across Environments
[PDF] [Poster]
2019 Proc. 16th the International Conference on Music Information Retrieval (ISMIR)
M. Fu, G. Xia, R. Dannenberg, and L. Wasserman A Statistical View on the Expressive Timing of Piano Rolled Chords.
[PDF] [Poster]
2015 Proc. 16th the International Conference on Music Information Retrieval (ISMIR)

Music Information Retrieval

Authors Title Year Book/Journal/Proceedings
G. Xia, D. Liang, R. Dannenberg, and M. Harvilla Segmentation, Clustering, and Display in a Personal Audio Database for Musicians.
[PDF] [Poster]
2011 Proc. 12th the International Conference on Music Information Retrieval (ISMIR)
G. Xia, T. Huang, Y. Ma, R. Dannenberg, and C. Faloutsos MidiFind: Similarity Search and Popularity Mining in Large MIDI Databases.
[PDF] [Website: online MIDI search]
2014 Sound, Music, and Motion, LNCS 2014, pp 259 - 276
J. Jiang, K. Chen, W. Li, and G. Xia Large Vocabulary Chord Transcription via Chord Structure Decomposition.
[PDF]
2019 Proc. 20th the International Conference on Music Information Retrieval (ISMIR)
Selected Music Event
Title Description Year Institution
Sally Garden Human-Robot Interactive Performances in collaboration with the world-leading saxphone robot [Video] and piano robot [Video]. Fall 2015 Carnegie Mellon, Waseda University, and TeoTronica Inc.
Laptop Orchestra
A collection of distributed performers who play laptop computers as instrument. [Webpage] [Poster] Summer 2012 Carnegie Mellon
DI & XIAO Solo Concert
Some representitive pieces of DI and XIAO were performed, including: [Video1] [Video2] [Video3] [Video4] [Video5] Summer 2010 Peking University

 

 

Web templates from Byron Boots: http://www.cc.gatech.edu/~bboots3/