Generative AI

10-423 + 10-623, Spring 2025
School of Computer Science
Carnegie Mellon University

Important Notes

This schedule is tentative and subject to change. Please check back often.

Tentative Schedule

Date Lecture Readings Announcements

Module
Weekday, Date Lecture Lecture # : Main Topic of Lecture

HW Due

HW Out

Generative models of text
Mon, 13-Jan Lecture 1 : RNN LMs / Autodiff
[Slides] [Slides (Inked)]

Wed, 15-Jan Lecture 2 : Transformer LMs
[Slides] [Slides (Inked)]

HW0 out

Fri, 17-Jan Recitation: HW0

Mon, 20-Jan (MLK Day - No Class)

Wed, 22-Jan Lecture 3 : Learning LLMs / Decoding
[Slides] [Slides (Inked)]

Fri, 24-Jan (No Recitation)

Mon, 27-Jan Lecture 4 : Pre-training, fine-tuning / Modern Transformers
[Slides] [Slides (Inked)]

HW0 due

HW1 out (L1-L4)

Generative models of images
Wed, 29-Jan Lecture 5 : Computer Vision: CNNs / Encoder-only Transformers / Vision Transformers
[Slides]

Quiz 1 (in-class, L1-L4)

Fri, 31-Jan Recitation: HW1

Mon, 3-Feb Lecture 6 : Generative Adversarial Networks (GANs) / PGM
[Slides] [Slides (Inked)]

Wed, 5-Feb Lecture 7 : Diffusion models (Part I)
[Slides] [Slides (Inked)]

Fri, 7-Feb (No Recitation)

Mon, 10-Feb Lecture 8 : Diffusion models (Part II) / Variational Autoencoders (VAEs)
[Slides] [Slides (Inked)]

HW1 due

HW2 out (L5-L8)

Applying and adapting foundation models
Wed, 12-Feb Lecture 9 : Variational Inference / In-context learning
[Slides] [Slides (Inked)]

Fri, 14-Feb Recitation: HW2

Mon, 17-Feb Lecture 10 : Parameter-efficient fine tuning
[Slides] [Slides (Inked)]

Quiz 2 (in-class, L5-L8)

Wed, 19-Feb Lecture 11 : Prompt Engineering / Instruction Fine-tuning / Reinforcement learning with human feedback (RLHF)
[Slides] [Slides (Inked)]

Project description out

Fri, 21-Feb Recitation: HW3

Sat, 22-Feb

HW2 due

Sun, 23-Feb

HW3 out (L9-L11)

Semester Course Drop Deadline

Multimodal foundation models
Mon, 24-Feb Lecture 12 : Direct Preference Optimization (DPO) / Text-to-image generation / Latent diffusion model
[Slides] [Slides (Inked)]

Wed, 26-Feb Lecture 13 : Prompt-to-Prompt
[Slides] [Slides (Inked)]

(Quiz 3 in-class, L9-L11)

Fri, 28-Feb (No Recitation)

Mon, 3-Mar Spring break

Tue, 4-Mar

Wed, 5-Mar Spring break

Thu, 6-Mar

Fri, 7-Mar Spring break

Mon, 10-Mar Lecture 14 : Vision-language models
[Slides] [Slides (Inked)]

Scaling Up
Wed, 12-Mar Lecture 15 : Scaling Laws
[Slides]

Thu, 13-Mar
[Handout]

HW3 due

HW4 out (L12-L14)

Fri, 14-Mar Recitation: HW4

Project team formation due by 2pm

Mon, 17-Mar Lecture 16 : Mixture of Experts
[Slides] [Slides (Inked)]

(Quiz 4 in-class, L12-L15)

Wed, 19-Mar Lecture 17 : Distributed training
[Slides]

Fri, 21-Mar (No Recitation)

Mon, 24-Mar Lecture 18 : Flash Attention / Efficient decoding strategies

HW4 due

HW623 out

Tue, 25-Mar

Advanced Topics
Wed, 26-Mar Lecture 19 : Long Context in LLM

Fri, 28-Mar (No Recitation)

Mon, 31-Mar In-Class Exam

Wed, 2-Apr Lecture 20 : Real-world Issues and Considerations (Part I) / What can go wrong?

Thu, 3-Apr (Spring Carnival - No Class)

Fri, 4-Apr (Spring Carnival - No Class)

Mon, 7-Apr Lecture 21 : Real-world Issues and Considerations (Part II)

(Quiz 5 in-class, L16-L20)

Tue, 8-Apr

Project proposal due

Wed, 9-Apr Lecture 22 : State Space Models

Fri, 11-Apr (No Recitation)

Mon, 14-Apr Lecture 23 : Code Generation / Autonomous Agents

Wed, 16-Apr Lecture 24 : TBD

Thu, 17-Apr

Project midway report due

Fri, 18-Apr (No Recitation)

Mon, 21-Apr Lecture 25 : Audio understanding and synthesis

HW623 due

Wed, 23-Apr Lecture 26 : Generative Models for Videos

Fri, 25-Apr (No Recitation)

Sun, 27-Apr

Project poster due

Thu, 1-May

Project final report due