10707 (Fall 2017): Deep Learning
Russ Salakhutdinov, Office: GHC 8017.
rsalakhu [at] cs [dot] cmu [dot] edu
- Lectures: Mon/Wed, 12:00 - 1:20pm, GHC 4401
- Office hours: Mondays, 11am-noon, GHC 8017
- TA Office hours:
- Hubert - Monday 4-5 pm at Gates 8th floor lounge area
- Otilia - Tuesday 3-4 pm at Gates 8021
- Dheeraj - Thursday 4-5 pm at Gates 5515
- Shunyuan - Friday 9-10 am at Gates 8th floor lounge area
- Extra office hours for A2: Week October 16th - October 20th
- Monday 9-10 am at GHC 8228 - Shunyuan
- Tuesday 3-4 pm at GHC 8228 - Otilia
- Wednesday 3-4 pm at GHC 8228 - Otilia
- Thursday 9-10 am at GHC 8228 - Shunyuan
- 3 assigments: 40%
- 1-hour midterm: 25%
- Final Project: 35%
This course covers some of the theory and methodology
of deep learning. The preliminary set of
topics to be covered
- Background: Linear Algebra, Distributions, Rules of probability.
- Regression, Classification.
- Feedforward neural nets, backpropagation algorithm.
Introduction to popular optimization and regularization techniques.
- Convolutional models with
applications to computer vision.
- Deep Learning Essentials
- Graphical Models: Directed and Undirected.
- Linear Factor Models, PPCA, FA, ICA,
Sparse Coding and its extensions.
- Autoencoders and its extensions.
Energy-based models, RBMs.
Monte Carlo Methods.
Learning and Inference:
Contrastive Divergence (CD), Stochastic
Maximum Likelihood Estimation, Score Matching, Ratio Matching,
- Annealed Importance Sampling,
Partition Function Estimation.
- Deep Generative Models:
Deep Belief Networks, Deep Boltzmann Machines,
Helmholtz Machines, Variational Autoencoders,
- Generative Adversarial Networks (GANs), Generative Moment Matching Nets,
Neural Autoregressive Density Estimator (NADE).
- Additional Topics
- More on Regularization and Optimization in Deep Nets.
- Sequence Modeling: Recurrent Neural Networks.
Sequence-to-Sequence Architectures, Attention models.
Deep Reinforcement Learning.
You can also use these books for additional reference:
Email: rsalakhu [at] cs [dot] cmu [dot] edu
Lecture Schedule |
10707 (Fall 2017): Topics in Deep Learning