Subject to change; check announcements and Canvas!.
| Class/Date | Class Topic | Reading Due | Resources |
|---|---|---|---|
| Part 0: Introduction Diagnostic problem set due on Tues, Jan 14. (TeX template and math_commands.tex macros file. ) | |||
| Class 1 (Tue, Jan 07) | Course logistics, introduction, failures of classical learning theory | Lecture Notes | |
| Class 2 (Thu, Jan 09) | Introduction to functional analysis, reproducing kernel Hilbert spaces | Lecture Notes | |
| Class 3 (Tue, Jan 14) | Double descent, implicit bias of gradient descent for regression | Fit Without Fear (Sections 1-3.9) (Required reading, but no reading summary due.) | Lecture Notes |
| Part 1: Overparameterized linear regression | |||
| Class 4 (Thu, Jan 16) | Introduction to high-dimensional asymptotics and random matrix theory | High Dimensional Regression (Section 2) | Lecture NotesDemo |
| Class 5 (Tue, Jan 21) | Effective regularization, risk of overparameterized ridge regression | High Dimensional Regression (Section 4) | Lecture Notes |
| Class 6 (Thu, Jan 23) | Benign overfitting | Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting (Section 2-3) | |
| Part 2: Approximating neural networks as kernel machines Final project intermediate check-in due on Tues, Feb 04. | |||
| Class 7 (Tue, Jan 28) | Infinite width NNs: Neural Tangent Kernel (Pt 1) | Deep Learning Theory Lecture Notes (Ch. 4) | |
| Class 8 (Thu, Jan 30) | Infinite width NNs: Neural Tangent Kernel (Pt 2) | Understanding the Neural Tangent Kernel | |
| Class 9 (Tue, Feb 04) | Feature learning, rich vs lazy regimes | The lazy (NTK) and rich (µP) regimes: A gentle tutorial (Optional reading; no reading summary due.) | |
| Part 3: Advanced topics | |||
| Class 10 (Thu, Feb 06) | Neural networks for classification | Deep Learning Theory Lecture Notes (Ch. 10) | |
| Class 11 (Tue, Feb 11) | Effects of depth | ||
| Class 12 (Thu, Feb 13) | Topic TBD (scaling laws, in-context learning, etc.) | ||
| Feb 14: oral presentations for final paper reading assignment | |||