Machine Learning for Structured Data

10-418 + 10-618, Fall 2022
School of Computer Science
Carnegie Mellon University


Important Notes

This schedule is tentative and subject to change. Please check back often.

Tentative Schedule

Date Lecture Readings Announcements

Search-based Structured Prediction

Mon, 29-Aug Lecture 1 : Course Overview / What is Structured Prediction?
[Slides] [Slides (Inked)] [Whiteboard (OneNote)]

Wed, 31-Aug Lecture 2 : Recurrent neural networks (RNNs) / Module-based Automatic Differentiation
[Slides] [Slides (Inked)] [Whiteboard (OneNote)]

Fri, 2-Sep (No Recitation)

Mon, 5-Sep (No Class: Labor Day)

Wed, 7-Sep Lecture 3 : 1D CNNs / Sequence-to-sequence Models
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]

HW1 out

Fri, 9-Sep Recitation: HW1
[Handout] [Whiteboard (OneNote)]

Mon, 12-Sep Lecture 4 : Learning to Search (Part I): MLE & Decoding for seq2seq / Imitation Learning
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]

Wed, 14-Sep Lecture 5 : Learning to Search (Part II): Imitation Learning / Structured Prediction as Search
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]

Fri, 16-Sep (No Recitation)

HW1 due

Sat, 17-Sep Lecture 5.5 : Learning to Search (Part III): Imitation Learning for Structured Prediction [video recorded]
[Slides] [Slides (Inked)] [Whiteboard (PDF)]

HW2 out

Graphical Models: Representation, Exact Inference, and Learning

Mon, 19-Sep Lecture 6 : Directed Graphical Models / Undirected Graphical Models
[Slides] [Whiteboard (OneNote)] [Poll]

Wed, 21-Sep Recitation: HW2
[Handout] [Whiteboard (OneNote)]

HW1 Solution Session

Fri, 23-Sep Lecture 7 : DGM & UGM Conditional Independencies / Factor Graphs
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]

Mon, 26-Sep Lecture 8 : Exact Marginal/MAP Inference: Variable Elimination & Belief Propagation
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]

Wed, 28-Sep Lecture 9 : Belief Propagation / Learning fully observable MRFs
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]

Thu, 29-Sep

HW2 due

HW3 out

Fri, 30-Sep (No Recitation)
[Whiteboard (OneNote)]

Mon, 3-Oct Lecture 10 : Learning fully observable CRFs / Neural Potential Functions / MBR Decoding
[Slides] [Slides (Inked)] [Whiteboard (OneNote)] [Poll]

HW2 Solution Session

Tue, 4-Oct Recitation: HW3 (evening)
[Handout] [Solutions]

Approximate Inference: MCMC

Wed, 5-Oct Lecture 11 : Complexity of Inference / Monte Carlo Methods
[Slides] [Slides (Inked)] [Whiteboard (OneNote)] [Poll]
  • Monte Carlo Methods. Li (2003). Information Theory, Inference, and Learning Algorithms, Chapter 29 (Section 29.1-29.3).

Fri, 7-Oct (No Recitation)

Mon, 10-Oct Lecture 12 : Midterm Exam Review / Markov Chain Monte Carlo: Gibbs Sampling & Metropolis-Hastings
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]
  • Monte Carlo Methods. Li (2003). Information Theory, Inference, and Learning Algorithms, Chapter 29 (Section 29.4 - 29.5).

HW3 due (only two grace/late days permitted)

Practice Exam out

Wed, 12-Oct Lecture 13 : Markov Chains / Bayesian Inference for Parameter Estimation
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Whiteboard (OneNote)] [Poll]
  • Monte Carlo Methods. Li (2003). Information Theory, Inference, and Learning Algorithms, Chapter 29 (Section 29.6 - 29.10).

Fri, 14-Oct Midterm Exam

Mon, 17-Oct Fall break

Tue, 18-Oct

Wed, 19-Oct Fall break

Thu, 20-Oct

Fri, 21-Oct Fall break

Mon, 24-Oct Lecture 14 : Bayesian Inference for Parameter Estimation / Topic Modeling
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]

HW4 out

Mon, 24-Oct Recitation: HW4 (evening recitation, 6pm, GHC 6121)
[Handout] [Solutions]

Wed, 26-Oct Lecture 15 : Topic Modeling / Convolutional Neural Networks
[Slides] [Slides (Inked)] [Poll]

Fri, 28-Oct Tartan Community Day

Approximate Inference: Variational Methods

Mon, 31-Oct Lecture 16 : Mean Field Variational Inference
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]

Wed, 2-Nov Lecture 17 : Coordinate Ascent Variational Inference
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]

Fri, 4-Nov Lecture 18 : CAVI / Expectation Maximization
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]

HW4 due

HW5 out

Mon, 7-Nov Recitation: HW5
[Handout]

Wed, 9-Nov Lecture 19 : Variational EM / Hidden State CRFs
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]

Fri, 11-Nov (No recitation)

Mon, 14-Nov Lecture 20 : Variational Autoencoders
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]

Advanced Topics

Wed, 16-Nov Lecture 21 : MAP Inference: Mixed Integer Linear Programming
[Slides] [Whiteboard (PDF)] [Poll]

HW5 due

HW6 out

Fri, 18-Nov Recitation: HW6
[Handout]

Mon, 21-Nov Lecture 22 : Structured Perceptron / Structured SVM
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]

Wed, 23-Nov Thanksgiving Holiday- No class

Thu, 24-Nov Thanksgiving Holiday- No class

Fri, 25-Nov Thanksgiving Holiday- No class

Mon, 28-Nov Lecture 23 : Causal Inference
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]

Mini-Project out

Wed, 30-Nov Lecture 24 : Causal Inference / Bayesian Nonparametrics
[Slides] [Slides (Inked)] [Poll]

HW6 due

Fri, 2-Dec (No recitation)

Mon, 5-Dec (Lecture rescheduled to Friday)

Practice Exam out

Wed, 7-Dec Lecture 25 : Bayesian Nonparametrics / Graph Neural Networks
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]

Fri, 9-Dec Lecture 26 : Graph Neural Networks / Final Exam Review
[Slides] [Slides (Inked)] [Whiteboard (PDF)] [Poll]

Mini-Project due

Thu, 15-Dec Final Exam (5:30 pm - 7:30 pm, DH A302)