Foundation models and generative AI
CS-461
This file is part of the content downloaded from Foundation models and generative AI.
- This course introduces the principles, methods, an... (Text and media area)
- Overview of Course Schedule: (Text and media area)
- All questions, discussions, and announcements will... (Text and media area)
- ed Forum for Announcements, Discussions, Questions (External tool)
- Getting Started with gNoto (File)
- Password for Recordings: CS-46125/6 (Text and media area)
Week 1: Introduction and Overview
- This week we will provide you an introduction to t... (Text and media area)
- Getting Ready (Quiz)
- Slides of Lecture 1 (File)
- Recording of Lecture 1 (URL)
- Paper: Sutton, Richard. "The Bitter Lesson." Incom... (Text and media area)
Week 2: Learning at Scale: Supervised, Self-Supervised, and Beyond
- This week we will cover supervised, unsupervised, ... (Text and media area)
- Lecture 2 (File)
- Exercise Session 1 - Recap (File)
- Exercise Sheet 1 (File)
- Exercise Sheet 1 - Solutions (File)
- Notebook 1 (File)
- Notebook 1 - Solutions (File)
- Paper: Kaplan et al., "Scaling Laws for Neural Lan... (Text and media area)
Week 3: Generative Models I: Autoregressive, Adversarial, and Autoencoder Models
- This week we explore three foundational approaches... (Text and media area)
- Lecture 3 (File)
- Recording of Lecture 3 (URL)
- Exercise Sheet 2 (File)
- Notebook 2 (File)
- Notebook 2 - Solutions (File)
- No readings this week. (Text and media area)
Week 4: Generative Models II: Diffusion Models and Beyond
- This week we explore denoising diffusion models th... (Text and media area)
- Guest Lecture 4 on Diffusion Models Foundations (File)
- Guest Lecture 4 on Diffusion Models Applications (File)
- Recording of Lecture 4 (URL)
- Exercise Sheet 3 (File)
- Exercise Sheet 3 - Solutions (File)
- Notebook 3 (File)
- Assignment 1 (Folder)
- Paper: Ho et al., "Denoising Diffusion Probabilist... (Text and media area)
Week 5: Tokenization Across Modalities and Building Blocks
- This week, we will first recap the chapter on gene... (Text and media area)
- Lecture 5 (File)
- Recording of Lecture 5 (URL)
- Exercise Sheet 4 (File)
- Exercise Sheet 4 - Solutions (File)
- Notebook 4 (File)
- Notebook 4 - Solutions (File)
- Paper: Vaswani et al., "Attention Is All You Need.... (Text and media area)
Week 6: Architectures I: Language and Vision Foundation Models
- After introducing the basic building blocks last w... (Text and media area)
- Guest Lecture 6 on Large Language Models and the Swiss Apertus (File)
- Lecture 6 on Vision FMs (File)
- Recording of Lecture 6 (URL)
- Assignment 1 (Assignment)
- Exercise Sheet 5 (File)
- Exercise Sheet 5 - Solutions (File)
- Notebook 5 (File)
- Notebook 5 - Solutions (File)
- Paper: Radford et al., "Improving Language Underst... (Text and media area)
Week 7: Semester Break
Week 8: Language FMs Continued
- Due to sickness, the lecture was cancelled this we... (Text and media area)
- Exercise Session Recap (File)
- Exercise Sheet 6 (File)
- Exercise Sheet 6 - Solutions (File)
- Notebook 6 (File)
- Notebook 6 - Solutions (File)
- Paper: Oquab et al., “DINOv2: Learning Robust Visu... (Text and media area)
Week 9: Architectures II: Foundation Models in the Sciences
- This week we will discuss concrete examples of fou... (Text and media area)
- Guest Lecture 9 on Foundation Models in Life Sciences and Medicine (File)
- Guest Lecture 9 on Foundation Models in Physics and Engineering (File)
- Recording of Lecture 9 (URL)
- Exercise Session 7 - Recap (File)
- Exercise Sheet 7 (File)
- Exercise Sheet 7 - Solutions (File)
- Notebook 7 (File)
- Notebook 7 - Solutions (File)
- No readings this week. (copy) (Text and media area)
Week 10: Multimodality in Foundation Models
- This week, we discuss how to build multimodal foun... (Text and media area)
- Lecture 10 (File)
- Recording of Lecture 10 (URL)
- Exercise Sheet 8 (File)
- Exercise Sheet 8 - Solutions (File)
- Notebook 8 (File)
- Notebook 8 - Solutions (File)
- Paper: Radford, Alec, et al. "Learning Transferabl... (Text and media area)
Week 11: Adaptation, Fine-Tuning, and Test-Time Training
- Next, we will discuss how foundation models can be... (Text and media area)
- Guest Lecture 11 on In-Context Learning and Test-Time Training of Foundation Models (File)
- Recording of Lecture 11 (URL)
- Assignment 2 (Folder)
- Exercise Sheet 9 (File)
- Notebook 9 (File)
- Notebook 9 - Solutions (File)
- Paper: Chen, Richard J., et al. "Towards a general... (Text and media area)
Week 12: World Models and Generative World Modeling
- So far, we have studied how foundation models can ... (Text and media area)
- Lecture 12 (File)
- Recording of Lecture 12 - Part 1 (Audio Only) (File)
- Recording of Lecture 12 - Part 2 (URL)
- Notebook 10 (File)
- Notebook 10 - Solutions (File)
- Paper: Liu, Bang, et al. "Advances and Challenges ... (Text and media area)
Week 13: Architectures III: Foundation Models in Robotics
- In Week 13, we turn to foundation models in roboti... (Text and media area)
- Guest Lecture 13 on Foundation Models in Robotics (File)
- Recording of Guest Lecture 13 (URL)
- Assignment 2 (Assignment)
- Exercise Sheet 11 (File)
- Exercise Sheet 11 - Solutions (File)
- Notebook 11 (File)
- Notebook 11 - Solutions (File)
Week 14: Foundation Models, Reinforcement Learning, Reasoning, and Decision-Making
- In week, we will focus on foundation models, reinf... (Text and media area)
- Lecture 14 (File)
- Recording of Lecture 14 (URL)
- Exercise Sheet 12 (File)
- Exercise Sheet 12 - Solutions (File)
- Notebook 12 (File)
- Notebook 12 - Solutions (File)
- Köbis et al., "Delegation to artificial intelligen... (Text and media area)
Week 15: Foundation Models and Agentic Systems and a Look into the Future
- We conclude with foundation models and agentic sys... (Text and media area)
- Lecture 15 (File)
- Recording of Lecture 15 (URL)
- Exercise Sheet 13 (File)
Recap Session for the Exam
- A recap session organized to answer your questions... (Text and media area)
- Slides (File)
- Recording of the recap session (File)