The Brain's Temporal Hierarchies
Dana Ballard, University of Rochester
The key to understanding the brain's direction of behavior will ultimately reside in the successful development of computational models. This tutorial focuses on a overview of brain architecture as composed of circuits that operate at different time scales.
MEMORIES. At the timescale of 80 milliseconds the cortex can be modeled as a content-addressable memory. Recent work from several different laboratories has shown that the synapses that mediate this memory can be entrained by presuming that the function of the memory is to predict its input with the minimal amount of hardware.
ROUTINES. At the timescale of 300 milliseconds the brain can be seen as a succession of behavioral routines that create temporary representations from multiple memory accesses. Recent simulations using video pipeline technology have shown how such routines can direct behavior in automobile driving.
BEHAVIORS. At the timescale of a few seconds, behavior can be seen as a succesion of eye and hand movements. The cooordination of such movements requires a minimal amount of state termed ``working memory.'' Recent experiments using head-mounted eye-trackers in virtual displays have shown that humans structure their eye and hand movements in order to minimize the amount of working memory that they use.
Dana Ballard obtained the PhD degree from the University of California at Irvine in 1974. During 1974-1975 he had a post-doctoral appointment at the Laboratorio Tecnologie Biomediche in Rome, Italy. Since 1975 he has been at the University of Rochester in the Computer Science Department where he has the rank of professor. He is the coauthor with Professor Brown of ``Computer Vision,'' (Prentice-Hall, 1982), and has recently completed a new text ``An Introduction to Natural Computation'' (MIT Press 1997) that is a general advanced introduction to mathematical models of the brain.
Ballard's current research focus is in computational theories of the brain that account for its real-time performance. In 1985 with Chris Brown, he led led a team that designed and built the first high speed binocular camera control system capable of simulating human saccadic eye movements. Recently he has extended his interests to the use of Virtual Reality equipment, both for robot modeling and human behavioral studies. He is the principal investigator in the National Institute of Health's Research Resource at Rochester, a set of multidisciplinary laboratories focusing on neural models of behavior.
The Support Vector Method (SVM) is a new general method of function estimation which does not depend explicitly on the dimensionality of the problem. It has been applied to pattern recognition, regression estimation, and density estimation problems as well as to solving linear operator equations.
In this tutorial the idea of the SVM, as well as some elements of its theory will be presented. In particular it will be shown that the generalization ability of the SVM is based on factors that classical statistics does not take into account. Therefore using the SVM one can generalize well in a high dimensional space on the basis of a small number of examples.
In the tutorial examples of solving various pattern recognition and regression estimation problems will be given and the results obtained will be compared with the results obtained using existing state-of-the-art techniques including neural networks.
Vladimir Vapnik, currently Member of Technical Staff, AT&T Labs-Research, is one of the creators of the theory of learning and generalization, the so-called VC theory (abbreviation for the Vapnik-Chervonenkis theory). This theory is a cornerstone for developing principles of inference from small sample sizes which can control the generalization ability of learning machines. On the basis of these principles he developed a new method for function estimation in high dimensional spaces, the so-called Support Vector Method. Vladimir Vapnik is the author of 7 monographs and more than 100 articles devoted to various problems of statistics and problems of learning and generalization.
High-level Vision and the Human Brain
Martha J. Farah, University of Pennsylvania
Object recognition, visual memory, and selective attention are difficult computational problems. I will review what is known about the solutions that the human brain evolved. My presentation will emphasize what can be learned from clinical case histories of brain-injured patients, and the recent results of functional neuroimaging, but will also invoke ideas and findings from animal neurophysiology and traditional cognitive psychology when particularly relevant.
Martha J. Farah took undergraduate degrees in Metallurgy and Philosophy from MIT, before earning a Ph.D. in Psychology from Harvard University. She has taught at Carnegie-Mellon University and at the University of Pennsylvania, where she is now Professor of Psychology. Her awards include the APA Early Career Contribution Award, the Troland Award from the National Academy of Sciences, and a Guggenheim Fellowship. Her books include: Visual Agnosia (1990), The Neuropsychology of High-Level Vision (edited with G. Ratcliff; 1994), and Behavioral Neurology and Neuropsychology (with T.E. Feinberg, 1997).
Graphical models are probabilistic graphs that have close relationships to neural networks. Classical undirected graphical models are closely related to Boltzmann machines. Directed graphical models (the more popular variety) are related to feedforward and recurrent neural networks, but have a stronger probabilistic semantics. Many interesting network models, including HMM's, mixture models, mixtures of experts models, Kalman filters, and general latent variable models can be viewed as special cases of graphical models. Graphical models are particularly well suited to unsupervised learning problems and to hybrids that mix unsupervised and supervised learning.
I will be describing basic graphical model inference algorithms (general algorithms for calculating the probability of unobserved nodes given the values of observed nodes), including the well-known junction tree algorithm. I will also describe sampling and variational methods for approximate inference. Finally, I will discuss various aspects of the problem of learning in the graphical model formalism, including the learning of parameters and the learning of structure.
Michael I. Jordan is Professor in the Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology. He received his Masters in Mathematics from Arizona State University, and earned his PhD in Cognitive Science from the University of California, San Diego. He has worked on a variety of topics in the area of machine learning, focusing on neural networks and graphical models. He has concentrated on the development of learning algorithms with statistical foundations, as exemplified by his work on the hierarchical mixture of experts (HME) model, a form of probabilistic decision tree. He has also worked on the general topic of variational approximations for graphical models. Professor Jordan is a member of the Center for Biological and Computational Learning at MIT, and is the recipient of a Presidential Young Investigator Award from the National Science Foundation.
Multineuronal Recording, Molecular Genetics, Behavior, and
Computation: New Approaches in the Study of Neural Function
Matthew A. Wilson, Massachusetts Institute of Technology
By introducing arrays of microelectrodes into cortical areas of freely behaving rodents, the coordinated activity of ensembles of large numbers of individual cells can be related to behavioral performance in tasks requiring sensory perception, discrimination, and memory. When combined with targeted molecular genetic manipulations, the role of specific biological mechanisms in these processes can be explored.
This tutorial will examine the use of these new approaches in the study of complex neural systems in the behaving animal.
Discussions will cover topics ranging from the technical aspects of multineuronal recording using multiple electrode arrays in hippocampal and neocortical regions, to the analysis of multineuronal data, current advances in molecular genetics relevant to the study of neural function, and behavioral paradigms used in the study of learning and memory in rodents.
Matthew A. Wilson received his Bachelors degree in Electrical Engineering at Rensselaer Polytechnic Institute and a Masters degree in Electrical Engineering at the University of Wisconsin, Madison. He entered the doctoral program in Computation and Neural Systems at the California Institute of Technology where he received his Ph.D. in 1990. During this period he authored the widely used GENESIS neural systems simulation software while carrying out his dissertation work on computer modeling of the olfactory cortex. During his postdoctoral training at the University of Arizona, Dr. Wilson developed the techniques for large-scale parallel recording of neuronal ensembles in behaving animals. He was recently appointed as Assistant Professor in the Department of Brain and Cognitive Sciences and the Biology Department at the Massachusetts Institute of Technology where he is pursuing behavioral neurophysiological research and computer modeling. Dr. Wilson was recently named an Alfred P. Sloan Research Fellow and appointed the Edward J. Poitras Assistant Professorship.
Recently, Neal, Williams, Rasmussen, Barber, Gibbs and MacKay have demonstrated that capabilities similar to those of feedforward neural networks are shown by simple statistical models called Gaussian processes.
This tutorial will give a simple introduction to Gaussian processes and their relationship to standard interpolation models. Various implementation methods (exact, approximate and Monte Carlo) will be reviewed, and we will discuss whether, for supervised regression and classification tasks, the feedforward network has been superceded.
David J.C. MacKay is a lecturer in the Department of Physics at Cambridge University. He completed his PhD on ``Bayesian Methods for Adaptive Models'' in the Computation and Neural Systems program at the California Institute of Technology in 1991. His interests include the construction and implementation of hierarchical Bayesian models that discover patterns in data, and the design and decoding of error correcting codes. In his free time he plays ultimate frisbee.
Please send comments and corrections to: