Painless Distribution Representations for Unsupervised Learning


Entropy and Mutual Information are popular tools for modeling data. In this talk I present alternative methods for comparing distributions which do not require density estimation but which rely on convergence properties of the expectation operator instead. This allows us to design algorithms for two-sample tests, independent component analysis, density estimation, clustering, feature selection, low-dimensional data representation and related unsupervised problems based on a unifying framework. It turns out that a large number of existing algorithms, ranging from sorting, k- means clustering, (kernel) principal component analysis, maximum variance unfolding, Pearson correlation to the Kolmogorov-Smirnov test and the earth mover's distance are special cases of our framework.

Speaker Bio

Alex Smola

Alex Smola is program leader of the Statistical Machine Learning Program at NICTA and adjunct full professor at the Computer Sciences Laboratory of the Australian National University. He has written over 100 refereed papers, edited four books and coauthored a book on 'Learning with Kernels'. He is member of the editorial boards of the Journal of Machine Learning, IEEE Pattern Analysis and Machine Intelligence, and Statistics and Computing. He has served on the program committees of NIPS, ICML and COLT and he has organized several Machine Learning Summer Schools. Dr Smola received his PhD in computer science from the University of Technology in Berlin in 1998 and his diplom in physics in 1996 from the University of Technology in Munich. Before joining the Australian National University in 2000 he worked at the Fraunhofer Institute in Berlin from 1996 to 1999 and at AT&T Research from 1995 to 1996.