Subject to change; check announcements and Canvas!.
| Class/Date | Class Topic | Reading Due | Resources |
|---|---|---|---|
| Part 0: Introduction Diagnostic problem set due on Tues, Jan 14. (TeX template and math_commands.tex macros file. ) | |||
| Class 1 (Tue, Jan 07) | Course logistics, introduction, failures of classical learning theory | Lecture Notes | |
| Class 2 (Thu, Jan 09) | Introduction to functional analysis, reproducing kernel Hilbert spaces | Lecture Notes | |
| Class 3 (Tue, Jan 14) | Double descent, implicit bias of gradient descent for regression | Fit Without Fear (Sections 1-3.9) (Required reading, but no reading summary due.) | Lecture Notes |
| Part 1: Overparameterized linear regression | |||
| Class 4 (Thu, Jan 16) | Introduction to high-dimensional asymptotics and random matrix theory | High Dimensional Regression (Section 2) | Lecture NotesDemo |
| Class 5 (Tue, Jan 21) | Effective regularization, risk of overparameterized ridge regression | High Dimensional Regression (Section 4) | Lecture Notes |
| Class 6 (Thu, Jan 23) | Benign overfitting | Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting (Section 2-3) | Lecture Notes |
| Part 2: Approximating neural networks as kernel machines Final project intermediate check-in due on Tues, Feb 04. | |||
| Class 7 (Tue, Jan 28) | Linear approximations of neural networks | Deep Learning Theory Lecture Notes (Ch. 4) | Lecture NotesDemo |
| Class 8 (Thu, Jan 30) | The neural tangent kernel approximation at initialization | (No reading; work on final project intermediate check-in) | Lecture Notes |
| Class 9 (Tue, Feb 04) | The neural tangent kernel during optmization | Understanding the Neural Tangent Kernel | Lecture Notes |
| Part 3: Advanced topics | |||
| Class 10 (Thu, Feb 06) | Neural networks for classification | Deep Learning Theory Lecture Notes (Ch. 10) | Lecture Notes |
| Class 11 (Tue, Feb 11) | Feature learning, rich vs lazy regimes | The lazy (NTK) and rich (µP) regimes: A gentle tutorial (Optional reading; no reading summary due.) | Lecture Notes |
| Class 12 (Thu, Feb 13) | My recent research (ensembles of neural networks) | Theoretical Limitations of Ensembles in the Age of Overparameterization (Optional reading; no reading summary due.) | |
| Feb 14: oral presentations for final paper reading assignment | |||