June 2008 @ Kobe, Japan

Kai-min Kevin Chang Research Associate (Special Faculty), Language Technologies Institute, School of Computer Science, Carnegie Mellon University. Affiliated with Center for Cognitive Brain Imaging, Center for the Neural Basis of Cognition Profile: CV, Resume Research Statement Email: kaimin dot chang at gmail dot com Phone: +1-412-268-1810 Fax: +1-412-268-6298 Office: Gates Hillman Center 5721 Address: Language Technologies Institute Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 USA

Research Interests

My research interests include using mathematical methodologies and machine learning techniques to investigate and model various human cognitive processes. In particular, I have studied semantic presentation of objects using functional Magnetic Resonance Imaging, knowledge representation in the context of an Intelligent Tutoring System, and language processing in the connectionist framework.

Recent advances in functional Magnetic Resonance Imaging (fMRI) provide a significant new approach to studying semantic representations in humans by making it possible to directly observe brain activity while people comprehend words and sentences. fMRI measures the hemodynamic response (changes in blood flow and blood oxygenation) related to neural activity in the human brain. Images can be acquired at good spatial resolution and reasonable temporal resolution - the activity level of 15,000 to 20,000 brain volume elements (voxels) of about 50 mm³ each can be measured every second. Supervised by Dr. Marcel Just and Dr. Tom Mitchell, I used functional Magnetic Resonance Imaging to study the cortical systems that underpin semantic representation of object knowledge. In a picture-naming task, participants were presented with black and white line drawings of 60 objects from a range of categories (e.g. tools, dwellings, animals, etc.) and were instructed to think of the same properties consistently during each presentation. In Mitchell et al. (2008), we showed that word features computed from the occurrences of stimulus words (within a trillion-token Google text corpus that captures the typical use of words in English text) can predict the brain activity associated with the 638 meaning of these words. We developed a generative model that is capable of predicting fMRI neural activity well enough that it can successfully match words it has not yet encountered to their previously unseen fMRI images with accuracies far above chance level. The distributed pattern of neural activity encodes the meanings of words, and the model's success indicates some initial access to the encoding. This work was also interviewed by CBS 60 Minutes: Mind Reading and aired on June 28, 2009. More recently, we applied the vector-based models of semantic representation used in computational linguistics to model neural activation patterns obtained while subjects comprehended multi-word expressions such as adjective-noun phrases (Chang et al., 2009) and noun-noun concept combinations.

Intelligent tutoring systems derive much of their power from having a student model that describes the learner's competencies. However, constructing a student model is challenging for computer tutors that use automated speech recognition (ASR) as input, due to inherent inaccuracies in ASR. Under the supervision of Dr. Jack Mostow and Dr. Joseph Beck, I proposed two models of developing word decoding skills and demonstrated that sufficient information existed in ASR output to determine which model better fits student performance and under what circumstances (Chang et al., 2005). Moreover, we found modeling individual learners' proficiencies may enable improved speech recognition in a computer tutor (Beck et al., 2005). In the above work, we used Knowledge Tracing, a derivative of Atkinson's model (1972) of human memory, to trace student's knowledge across different skills. We then followed Reye's work (1998), which proved that Knowledge Tracing is a special case of a Bayesian network, and implemented a generic Bayesian network toolkit (BNT-SM; Chang et al., 2006) for student modeling. BNT-SM inputs a data set and a compact XML specification of a (dynamic) Bayes net model hypothesized by a researcher to describe causal relationships among student knowledge and observed behavior. It generates and executes the code to train and test the model using the Bayes Net Toolbox (Murphy, 1998). BNT-SM allows researchers to easily explore different hypothesis with respect to the knowledge representation in a student model. For example, by varying the graphical structure of a Bayesian network, we examined how tutoring intervention can affect students' knowledge state - whether the intervention is likely to scaffold or to help students to learn.

I have also worked on low-level computational models of reading, in which cognitive processes are implemented in terms of cooperative and competitive interactions among large numbers of simple, neuron-like processing units. Contemporary leaders in computational model of reading are divided in whether a localist or distributed representation is more appropriate. For example, localist representation assumes activation of a word to be a corresponding lexical unit, whereas distributed representation assumes such to be a pattern of activation distributed over a number of primitive representational units. Fortunately, I have the opportunity to work in both camps. For the localist representation, I worked with Dr. Derek Besner to challenge one of the fundamental assumption of DRC, a leading computational model of reading which assumes a Dual Route model and that information processing occurs in a Cascaded fashion within the model. In order to have the model correctly reproduces the joint effects of letter length and stimulus quality seen in skilled readers, I implemented a threshold at the letter level in DRC, as part of my undergraduate thesis. For the distributed representation, I worked with Dr. David Plaut to address two tasks in the reading model of parallel distributed processing (PDP) framework: 1) learning static representations of variable-length strings, and 2) generating continuous articulatory trajectories as output. These two tasks are fundamental extensions of PDP modeling of word reading, enabling models to process multi-syllabic words and to generate more realistic analogues of human response time.

Current Projects

The ultimate automated tutor could peer directly into students' minds to identify their mental states (e.g. engagement, competencies, and intentions) and decide accordingly what and how to teach at each moment. Recent advances in brain imaging technologies have brought upon several portable EEG headsets that are commercially-available and simple enough to use in schools (NeuroSky; Emotiv; BCInet). Using EEG signals recorded from adults and children reading text and isolated words, both aloud and silently, we train and test classifiers to tell if students are reading easy or hard sentences, and to distinguish among easy words, hard words, pseudo-words, and unpronounceable strings (Mostow, Chang & Nelson, 2011). Better-than-chance performance shows promise for tutors to use EEG at school. This development makes it feasible to record longitudinal EEG data in authentic school settings.

Humans use speech to communicate what's on their mind. However, until now, automatic speech recognizers (ASR) and dialogue systems have had no direct way to take into account what is going on in a speaker's or listener's mind. One way to address this limitation is to use EEG signals to infer mental states. Chen et al. (to appear) utilized EEG to adapt language model for ASR. We train and test classifiers that input this EEG signal from adults and children reading text. We use its probabilistic output to control weighted interpolation of separate language models for easy and difficult reading. We show that such EEG-adapted ASR achieves higher accuracy than two baselines, and analyze how its performance depends on EEG classification accuracy. Furthermore, ASR are usually used to recognize a target speaker's speech, but sometimes they are used in an environment full of other sounds (e.g., background noise, speech from other people, etc.). The ASR recognition performance suffers as some of the background noises are erroneously recognized as user speech and the insertion errors increases as a result. Because humans use speech to communicate what's on their mind, taking into account speakers' mind can help distinguish if the sounds are from the speaker's speech or not. Chen et al. (submitted) train and test EEG classifiers that estimate the probability that a target user is speaking, listening or idling. The probability estimates are used to remove recognized words not from the target user and thereby reduce insertion errors generated by other sounds. These pilot work are steps towards improving ASR more generally by using EEG to distinguish mental states.

Academic Activities

• Co-organizing NIPS 2012 Workshop on Machine Learning and Interpretation in Neuroimaging, Lake Tahoe, NV, 2012 - 2013.
• Co-PIed on NSF 10-620 Cyberlearning: Tranforming Education grant (DIP: Exploiting Longitudinal EEG Input in a Reading Tutor), 2011-2014.
• Co-organized NIPS 2011 Workshop on Machine Learning and Interpretation in Neuroimaging, Granada, Spain, 2011 - 2012.
• Invited talk at Brain Informatics, Special Session on Computational Neurolinguistics, Toronto, Canada, 2010.
• Co-organized NAACL-HLT 2010 Workshop on Computational Neurolinguistics, Los Angeles, CA, 2009 - 2010.

Software Tools

• Real-time EEG brain activity (MindSet).
• Distribution of the 60 Word-Pic fMRI data used in the Science paper (Science 2008 Data).
• 4CAPS Cognitive Neuroarchitecture (4CAPS).
• Bayes Net Toolbox for Student Modeling (BNT-SM).

Demo

• EEG-enabled MOOC (1).
• NeuroDialog (1, 2).
• Mind Reading (CBS 60 Minutes: Mind Reading).

Journal Publications

1. Chang, K.M., Mitchell, T.M., & Just M.A. (2010). Quantitative modeling of the neural representations of objects: How semantic feature norms can account for fMRI activation. NeuroImage: Special Issue on Multivariate Decoding and Brain Reading, 56, 716-727. (pdf, doi)
2. Kemp, C., Chang, K.M., & Lombardi, L. (2010). Category and feature identification. Acta Psychologica, 133, 216-233. (pdf)
3. Mitchell, T.M., Shinkareva, S.V., Carlson, A., Chang, K.M., Malave, V.L., Mason, R.A., & Just, M.A. (2008). Predicting human brain activity associated with noun meanings. Science, 320, 1191-1195. (pdf, supporting material)

Peer-Reviewed Conference Proceedings

1. Sridharan, S., Chen, Y.N., Chang, K.M., & Rudnicky, A. (2012). NeuroDialog: An EEG-Enabled Spoken Dialog Interface. Proceedings of the 14th ACM International Conference on Multi-modal Interaction, Santa Monica, CA, 65-66. (pdf)
2. Chen, Y.N., Chang, K.M., & Mostow, J. (2012) Towards using EEG to improve ASR accuracy. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Montreal, Canada, 377-381. (pdf)
3. Chang, K.M., Murphy, B., & Just, M.A. (2012). A Latent Feature Analysis of the Neural Representation of Conceptual Knowledge. Proceedings of the 25th Advances in Neural Information Processing Systems Workshop on Machine Learning and Interpretation in Neuroimaging, Granada, Spain, Lecture Notes in Computer Science, 7263, 244-255. (pdf)
4. Mostow, J., Chang, K.M., & Nelson, J. (2011). Toward exploiting EEG input in a Reading Tutor. Proceedings of the 15th International Conference on Artificial Intelligence in Education, Auckland, New Zealand, 230-237. (pdf; Best Paper Nominee)
5. Chang, K.M., Cherkassky, V.L., Mitchell, T.M., & Just M.A. (2009). Quantitative modeling of the neural representation of adjective-noun phrases to account for fMRI activation. Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, Suntec, Singapore, 638-646. (pdf)
6. Jern, A., Chang, K.M., & Kemp, C. (2009). Bayesian belief polarization. Advances in Neural Information Processing Systems, Vancouver, Canada, 22. (pdf, supporting material)
7. Beck, J.E., Chang, K.M., Mostow, J., & Corbett, A. (2008). Does help help? Introducing the Bayesian evaluation and assessment methodology. Proceedings of the 9th International Conference on Intelligent Tutoring Systems, Montreal, Canada, 383-394. (pdf, Best Paper Award)
8. Beck, J.E. & Chang, K.M. (2007). Identifiability: A fundamental problem of student modeling. Proceedings of the 11th International Conference on User Modeling, Corfu, Greece, 137-146. (pdf)
9. Chang, K.M., Beck, J.E., Mostow, J., & Corbett, A. (2006). A Bayes net toolkit for student modeling in intelligent tutoring systems. Proceedings of the 8th International Conference on Intelligent Tutoring Systems, Jhongli, Taiwan, 104-113. (pdf)
10. Chang, K.M., Beck, J.E., Mostow, J., & Corbett, A. (2006). Does help help? A Bayes net approach to modeling tutor interventions. Proceedings of the 21st National Conference on Artificial Intelligence Workshop on Educational Data Mining, Boston, MA, 41-46. (pdf)
11. Chang, K.M., Beck, J.E., Mostow, J., & Corbett, A. (2005). Using speech recognition to evaluate two student models for a Reading Tutor. Proceedings of the 12th International Conference on Artificial Intelligence in Education Workshop on Student Modeling for Language Tutors, Amsterdam, Netherlands, 12-21. (pdf)
12. Beck, J.E., Chang, K.M., Mostow, J., & Corbett, A. (2005). Using a student model to improve a computer tutor's speech recognition. Proceedings of the 12th International Conference on Artificial Intelligence in Education Workshop on Student Modeling for Language Tutors, Amsterdam, Netherlands, 2-11. (pdf)

Poster Presentations

1. Chang, K.M., Mitchell, T.M., & Just M.A. (2008). What functional brain imaging reveals about neuroarchitecture of object knowledge. The 30th Annual Meeting of the Cognitive Science Society, Washington, DC.
2. Chang, K.M., Malave V., Shinkareva, S., Mitchell, T.M., & Just M.A. (2007). What functional brain imaging reveals about neuroarchitecture of object knowledge. The 1st Okinawa Institute of Science and Technology Workshop on Cognitive Neurobiology, Okinawa, Japan.

Thesis

1. Chang, K.M. (2010). Quantitative modeling of the neural representation of nouns and phrases. Ph.D. Proposal. (pdf) (oral) Ph.D. Thesis. (pdf, oral)
2. Chang, K.M. (2003). Evolution of the dual route cascaded model of reading aloud. Undergraduate Thesis. (pdf)

Professional Activities

Biographical Sketch

Personal Interest

Some people write their diaries with words, some record them with pictures. I mark mine with food! Yes, I love to eat! My plan is taste all the savoury dishes in the world and mark them on my Savoury Google Maps! Still a long way to go, but I am getting there! :p I like to read the Slashdot, the tw.bbs.talk.joke newsgroup, and watch Comedy Central on TV. Three comic strips that I frequently visit are Piled Higher and Deeper, Dilbert, and River's 543. For leisure, I enjoy playing poker, chess and pool. I am also very into mobile devices. I frequent xda-developers and stay up to date on many smart phone devices. My current phone is AT & T Tilt2. Finally, I treasure freedom in speech, thoughts, codes, and am an advocate of Open Source software. PS, I was named a student of Watermelon according to this news article, originally published by University of Waterloo school officials on Apr 1, 2003. ;) Quite frankly, I joined Carnegie Mellon University later and that indeed made me a Watermelon. FYI, Kevyn Collins-Thompson is also a Watermelon.