Nihar B. Shah

10-715 Fall 2025: Advanced Introduction to Machine Learning

This course is designed for Ph.D. students whose primary field of study is machine learning, or who intend to make machine learning methodological research a main focus of their thesis. It will give students a thorough grounding in the algorithms, mathematics, theories, and insights needed to do in-depth research and applications in machine learning. The topics of this course will in part parallel those covered in the general graduate machine learning course (10-701), but with a greater emphasis on depth in terms of theory (mathematical guarantees and proofs).

IMPORTANT NOTE: Students entering the class are expected to have a pre-existing strong working knowledge of linear algebra (e.g., mathematical representation of subspaces, singular value decomposition), probability (e.g., multivariate Gaussians, Bayes' rule, conditional expectation), and calculus (e.g., derivative of a vector with respect to another vector), and programming (we'll mostly be supporting Python). This class is best for you if you have machine learning at the CORE of your studies/research, and want to understand the fundamentals. This class will be HEAVY and will move FAST. If machine learning is an auxiliary component of your studies/research or if you do not have the required background, you may consider the general graduate Machine Learning course (10-701) or the Masters-level Machine Learning course (10-601). Click here for an ML course comparison. Note that machine learning itself is NOT a prerequisite for this course.
Also note that by departmental policy, this course is open only to graduate students (5th year masters are allowed).

Waitlist: The waitlist will be processed near end of summer.

Time and location: The class is scheduled for Monday and Wednesday 2pm to 3.20pm in POS 152. We will reserve Fridays for recitations and makeup classes, which will be held 2pm to 3.20pm in the same location.

Units: 12

Instructor: Nihar B. Shah

Grading and other logistics: Will be discussed in the first lecture.

Textbook (for part of the course): [SB] Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David (available online)

Syllabus and lecture schedule (slightly tentative, mostly fixed). Note that most lectures will be taught on the board.

Date	Topic
Monday, Aug 25	Logistics and introduction to introduction to ML
Wednesday, Aug 27	Perceptrons, linear hypothesis class, support vector machines
Friday, Aug 29	Kernel methods, multi-layer perceptrons
Monday, Sep 1	Labor day
Wednesday, Sep 3	Recitation: Optimization for ML
Friday, Sep 5	No class
Monday, Sep 8	History of ML 1 (Tom Mitchell)
Wednesday, Sep 10	History of ML 2 (Tom Mitchell)
Friday, Sep 12	Recitation: Linear regression, Logistic regression
Monday, Sep 15	Learning theory 1
Wednesday, Sep 17	Learning theory 2
Friday, Sep 19	Recitation: Tail bounds
Monday, Sep 22	Learning theory 3
Wednesday, Sep 24	Learning theory 4
Friday, Sep 26	Recitation: MLE and MAP
Monday, Sep 29	Neural networks representation power, CNNs, Resnets
Wednesday, Oct 1	Midterm
Friday, Oct 3	No class
Monday, Oct 6	Neural networks, model selection, bias-complexity tradeoff, interpolation regime
Wednesday, Oct 8	Multi-armed bandits, Reinforcement learning 1
Friday, Oct 10	No class
Oct 13-17	Fall break
Monday, Oct 20	No class
Wednesday, Oct 22	No class
Friday, Oct 24	Reinforcement learning 2
Monday, Oct 27	Reinforcement learning 3; Graphical models and Causality 1
Wednesday, Oct 29	Graphical models and Causality 2
Friday, Oct 31	No class
Monday, Nov 3	Language models, attention mechanisms, transformers (Zico Kolter)
Wednesday, Nov 5	Language models, attention mechanisms, transformers (Zico Kolter)
Friday, Nov 7	Language models, attention mechanisms, transformers (Zico Kolter)
Monday, Nov 10	Language models, attention mechanisms, transformers (Zico Kolter)
Wednesday, Nov 12	Language models, attention mechanisms, transformers (Zico Kolter)
Friday, Nov 14	Language models, attention mechanisms, transformers (Zico Kolter) [Only one day out of Nov 7 and 14 will be used]
Monday, Nov 17	Diffusion models and GANs (Barnabas Poczos)
Wednesday, Nov 19	Diffusion models and GANs (Barnabas Poczos)
Friday, Nov 21	Diffusion models and GANs (Barnabas Poczos)
Monday, Nov 24	Diffusion models and GANs (Barnabas Poczos)
Nov 26-28	Thanksgiving
Monday, Dec 1	Diffusion models and GANs (Barnabas Poczos)
Wednesday, Dec 3	Diffusion models and GANs (Barnabas Poczos)
Finals week	Final exam (university will announce exact date/time)

Accommodations for Students with Disabilities: If you have a disability and have an accommodations letter from the Disability Resources office, I encourage you to discuss your accommodations and needs with me as early in the semester as possible. I will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the Office of Disability Resources, I encourage you to contact them at access@andrew.cmu.edu.