10707 (Fall 2017): Deep Learning

Instructor :
  • Russ Salakhutdinov, Office: GHC 8017.
  • Email: rsalakhu [at] cs [dot] cmu [dot] edu

  • Lectures: Mon/Wed, 12:00 - 1:20pm, GHC 4401
  • Office hours: The week of: Monday, 2:30pm-3:30pm, GHC 8017
  • TA Office hours:
    • Hubert - Monday 4-5 pm at Gates 8th floor lounge area
    • Otilia - Tuesday 3-4 pm at Gates 8021
    • Dheeraj - Thursday 4-5 pm at Gates 5515
    • Shunyuan - Friday 9-10 am at Gates 8th floor lounge area

    Marking Scheme:

    • 3 assigments: 40%
    • 1-hour midterm: 25%
    • Final Project: 35%

Course Outline:

This course covers some of the theory and methodology of deep learning. The preliminary set of topics to be covered include:
  • Introduction
    • Background: Linear Algebra, Distributions, Rules of probability.
    • Regression, Classification.
    • Feedforward neural nets, backpropagation algorithm. Introduction to popular optimization and regularization techniques.
    • Convolutional models with applications to computer vision.

  • Deep Learning Essentials
    • Graphical Models: Directed and Undirected.
    • Linear Factor Models, PPCA, FA, ICA, Sparse Coding and its extensions.
    • Autoencoders and its extensions. Energy-based models, RBMs.
    • Monte Carlo Methods.
    • Learning and Inference: Contrastive Divergence (CD), Stochastic Maximum Likelihood Estimation, Score Matching, Ratio Matching, Pseud-likelihood Estimation, Noise-Contrastive Estimation.
    • Annealed Importance Sampling, Partition Function Estimation.
    • Deep Generative Models: Deep Belief Networks, Deep Boltzmann Machines, Helmholtz Machines, Variational Autoencoders, Importance-weighted Autoencoders, Wake-Sleep Algorithm.
    • Generative Adversarial Networks (GANs), Generative Moment Matching Nets, Neural Autoregressive Density Estimator (NADE).

  • Additional Topics
    • More on Regularization and Optimization in Deep Nets.
    • Sequence Modeling: Recurrent Neural Networks. Sequence-to-Sequence Architectures, Attention models.
    • Deep Reinforcement Learning.

Books :
You can also use these books for additional reference:

Contact Information

Email: rsalakhu [at] cs [dot] cmu [dot] edu

[ Home | Assignments | Lecture Schedule | ]

10707 (Fall 2017): Topics in Deep Learning || http://www.cs.cmu.edu/~rsalakhu/10707/