
10707 (Fall 2017): Deep Learning
Instructor :

Russ Salakhutdinov, Office: GHC 8017.
 Email:
rsalakhu [at] cs [dot] cmu [dot] edu
 Lectures: Mon/Wed, 12:00  1:20pm, GHC 4401
 Office hours: The week of: Monday, 2:30pm3:30pm, GHC 8017
 TA Office hours:
 Hubert  Monday 45 pm at Gates 8th floor lounge area
 Otilia  Tuesday 34 pm at Gates 8021
 Dheeraj  Thursday 45 pm at Gates 5515
 Shunyuan  Friday 910 am at Gates 8th floor lounge area
Marking Scheme:
 3 assigments: 40%
 1hour midterm: 25%
 Final Project: 35%
Course Outline:
This course covers some of the theory and methodology
of deep learning. The preliminary set of
topics to be covered
include:
 Introduction
 Background: Linear Algebra, Distributions, Rules of probability.
 Regression, Classification.
 Feedforward neural nets, backpropagation algorithm.
Introduction to popular optimization and regularization techniques.
 Convolutional models with
applications to computer vision.
 Deep Learning Essentials
 Graphical Models: Directed and Undirected.
 Linear Factor Models, PPCA, FA, ICA,
Sparse Coding and its extensions.
 Autoencoders and its extensions.
Energybased models, RBMs.

Monte Carlo Methods.

Learning and Inference:
Contrastive Divergence (CD), Stochastic
Maximum Likelihood Estimation, Score Matching, Ratio Matching,
Pseudlikelihood Estimation,
NoiseContrastive Estimation.
 Annealed Importance Sampling,
Partition Function Estimation.
 Deep Generative Models:
Deep Belief Networks, Deep Boltzmann Machines,
Helmholtz Machines, Variational Autoencoders,
Importanceweighted Autoencoders,
WakeSleep Algorithm.
 Generative Adversarial Networks (GANs), Generative Moment Matching Nets,
Neural Autoregressive Density Estimator (NADE).
 Additional Topics
 More on Regularization and Optimization in Deep Nets.
 Sequence Modeling: Recurrent Neural Networks.
SequencetoSequence Architectures, Attention models.

Deep Reinforcement Learning.
Books :
You can also use these books for additional reference:
Contact Information
Email: rsalakhu [at] cs [dot] cmu [dot] edu
[
Home 
Assignments 
Lecture Schedule 
]
10707 (Fall 2017): Topics in Deep Learning
 http://www.cs.cmu.edu/~rsalakhu/10707/
