Automatic Audio Analysis
Polyphonic structural analysis of music by using adaptive Dynamic Programming on audio (spectrum, chroma) and / or symbolic (MIDI) representations
Design and implement a valid Hidden Markov Model for precise audio alignment
Design a Neural Net Model to find better audio features for audio analysis.
- Application prototypes
- Polyphonic accompniment
- Audio information retrieval
- Intelligent audio editor
- Expressive performance analysis
Past Projects but Still Alive
- Music Information Retrieval (MIR)
- Investigated and designed the retrieval techniques for "Query-by-Humming" IR system and algorithms (Dynamic Programming and Hidden Markov Model) for melodic similarity comparison
- Collaborated to design interactive interfaces for future, intelligent music browsers and databases
- Nyquist is a sound synthesis & composition language invented by Roger B. Dannenberg.
It is based on a Lisp syntax. Nyquist is an elegant and powerful system based on functional programming
- Implemented a piano synthesizer in Nyquist, which is one of the most commonly used Nyquist synthesizer. For detail reference see M4C
- Designed and developed an windows IDE for Nyquist. So far it is the most popular Nyquist IDE
- Took part in the work of debugging Nyquis
Spring, 2003 Graduate Database System course project Carnegie Mellon University
- Ning Hu and Minglong Shao, "Kalman-Tree: an Index Structure on Spatio-Temporal Data"
- The project proposes a modified R-tree indexing structure (KR-tree) targeting the future location estimation problem in spatio-temporal databases, by integrating the useful estimator Kalman filter.
Fall, 2002 Grad. Computer Architecture course project Carnegie Mellon University
2002 Summer Research Internship Language Technology Institute, CMU
- Took part in the development of JAVELIN , an open-domain question answering system, and prepared it for the TREC 2002 question-answering track.
- Solely responsible for the development and improvement on one of the main components Request Filler, which identifies and scores answers from a set of potentially relevant documents. Major contributions include component expansion for new answer types, feature vectors evaluation and selection, decision tree classifier (c4.5) training and improvement, new classifier (K-Nearest-Neighbor implementing KD-trees) development, data and result analysis, module maintenance, etc.
Spring, 2002 Computer Animation course project Carnegie Mellon University
- Ning Hu, Philo Chua, Kevin Auyoung, "Musically Interactive Rendered Animation (MIRA)" [Video: 70MB]
- Built an animated figure that dances on stage with the rhythm / music using an existing set of motion capture data. Main techniques involved are Motion Capture, Motion Blending, Beat Tracking, Non-Photorealistic Rendering
2001 - 2002 Motion Capture project Entertainment Technology Center, CMU
- Training for Physical Tasks in Virtual Environments: Tai Chi
- Collaborated to develop a wireless virtual reality system that combines full-body real-time motion capture and wireless head-mounted display. A prototype full-body Tai Chi training application was also built with it
- Solely responsible for Researching and implementing real-time filters (damping filter, low pass filter, Kalman filter, etc.) for noisy motion capture data, and analyzing Tai Chi experiment data through various statistical approaches
2001 Earth Theater project Entertainment Technology Center, CMU
- Collaborated to create an interactive and educational virtual reality show for the Earth Theater at the Carnegie Museum of Natural History, Pittsburgh
- Personally in charge of sound design and the implementation of a networking Client / Server program dealing with spatialized audio effect
2000 "Jam-O-Drum" project Entertainment Technology Center, CMU
- Collaborated to develop new experiences for "Jam-O-Drum", a musical interaction device invented by Tina Blaine(Bean). Our project was exhibited on both SIGGRAPH 2001 and INTERFACE 2001.
- My contributions include brain-storming and developing interactive games for "Jam-O-Drum" on Macintosh in C++, music composition and sound design
Last modified: Apr 11, 2004