Date: Wednesday, 20-Nov-96 22:25:30 GMT Server: NCSA/1.3 MIME-version: 1.0 Content-type: text/html Last-modified: Sunday, 28-Aug-94 01:57:28 GMT Content-length: 3479 Prof Abu-Mostafa

Yaser S. Abu-Mostafa

Professor of Electrical Engineering and Computer Science.

B.Sc. 1979, Cairo University;

M.S.E.E. 1981, Georgia Institute of Technology;

Ph.D. 1983, California Institute of Technology.

The Learning Systems group at Caltech works on the theory, implementation, and application of automated learning, pattern recognition, and neural networks. We are an interdisciplinary group with students coming from Electrical Engineering, Computer Science, Mathematics, and Physics. We work on a variety of projects analyzing and synthesizing systems that can be trained to perform their task.

Learning is a topic that can be approached from different angles. For instance, one project deals with the rigorous mathematical analysis of over-learning; when a system memorizes too much detail about a limited experience. Another project develops techniques for selecting high-level features for pattern recognition. A third project applies learning to forecasting in the financial markets.

We have an on-going research effort in learning from hints which generalizes the basic learning from examples; paradigm. Learning from examples assumes that the function we are trying to learn is represented to us by a training set of input/output examples, but is otherwise unknown. This poses a dilemma: We would like to use a learning model that is sophisticated enough to have a chance of implementing the unknown function, yet we want the model to be simple enough that a limited set of examples will suffice to tune it properly.

Hints come in as a learning aid to improve the situation. We usually have, in addition to the set of examples of , some prior knowledge of certain facts about . We wish to use this side information to our advantage. For instance, we may know that is scale-invariant and monotonic, and we want to put this information together with the training set of examples to come up with a good implementation of . We want this process to be automated, so that the system is genuinely learning from hints.

Our group has pioneered the use of hints in learning, and we continue to work on all aspects of the technique. The goal is to automate the use of hints to a degree where we can effectively utilize a large number of different hints that may be available in a practical situation. We have also developed a theoretical analysis of learning from hints. The analysis is based on the Vapnik-Chervonenkis dimension provides an estimate for the number of examples the system will need to learn .

The Learning Systems group is committed to the understanding of the fundamental components of automated learning, and to the development of real-life systems that utilize learning to achieve state-of-the art performance.

Selected Publications

Recognitive Aspects of Moment Invariants, Y. Abu-Mostafa and D. Psaltis, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. PAMI-6, 698-706, November 1984.

Information Capacity of the Hopfield Model, Y. Abu-Mostafa,

IEEE Trans. on Information Theory, Vol. IT-32, 513-525, July 1986.

Learning from Hints in Neural Networks, Y. Abu-Mostafa, Journal of complexity, Vol. 6, 192-198, June 1990.

Hints and the VC Dimension, Y. Abu-Mostafa, Neural Computation, Vol. 5, 278-288, MIT Press, March 1993.