Deep learning

This course explores how to design reliable discriminative and generative neural networks, the ethics of data acquisition and model deployment, as well as modern multi-modal models.

This course has a fixed capacity and is now fully booked. We are unable to accept any new registered participants. Thank you for your understanding.

Lectures and labs

Course content

This course equips students with a comprehensive foundation of modern deep learning, enabling students to design and train discriminative and generative neural networks for a wide range of tasks. Topics include:

• Deep learning applications (natural language processing, computer vision, audio processing, biology, robotics, science), principles and regulations
• Loss functions, data and labels, data provenance
• Training models: gradients and initialization
• Generalization and performance
• Transformers
• Graph neural networks
• Multi-modal models
  
• Generative adversarial networks
• Variational autoencoders
• Diffusion models
• Interpretability, explanations, bias and fairness
  
  

Learning prerequisites

• Basics in probabilities and statistics
• Linear algebra
• Differential calculus
• Python programming

• Basics in optimization
• Basics in algorithmic
• Basics in signal processing

Important concepts to start the course

Discrete and continuous distributions, normal density, law of large numbers, conditional probabilities, Bayes, PCA, vector, matrix operations, Euclidean spaces, Jacobian, Hessian, chain rule, notion of minima, gradient descent, computational costs, Fourier transform, convolution.

Instructor

Andrea Cavallaro

Teaching Assistants

Olena Hrynenko
Darya Baranouskaya
Ti Wang
Qin Liu
Egor Rumiantsev
Corentin Genton

Exercises

Topics

Exercises

Topics

Exercises

Topics

Topics

Natural Language Processing: how to analyse / synthesize language
Tokens: on breaking text into small units
Self-attention: how a model can focus on relevant parts of the input
Transformer: how to use attention in a neural network
Encoder model: how to process text to create a useful representation
Decoder model: how to generate text
Encoder-decoder model: how to map a sequence to a sequence
Exercises: positional encoding, attention, and transformers

Exercises

Topics

Graphs: on nodes, edges and structure
Simple graph: on aggregation and parameter sharing
Tasks on graphs: how to perform regression and classification
Graph convolutional networks: on deep learning with graphs
Graph attention: on weighted, learned, neighbor feature aggregation
Training: how to deal with the structure
Line graphs: on the complementary graph
Graph types: on the diversity of graph representations
Exercises: message passing and graph classification

Exercises

Slides | Practice | Solutions | Questions

Topics

Topics

Flamingo: vision and language model example
Vision-Language-Action models: robots that follow instructions
Reinforcement learning from human feedback: on value alignment
Pluralistic alignment: on AI respecting the values of all

Slides | Video

Lab

Topics

Lab

Topics

Generative Adversarial Networks: on synthetizing data from noise
Image-to-image GANs: on translating images across domains
StyleGAN: on creating hyperrealistic images with control of details
Exercises: implementing and training a GAN

Slides | Video

Lab

Topics

As we are in the final weeks of the group project, this lecture focuses on recommendations for the completion of the project, covering the critical analysis of the results, benchmarking, poster and report preparation, and poster presentation. The aim is to support you in these final steps and maximize your learning.

Topics

Learning outcomes (recap!)
Project deliverables
Assessment criteria
Poster and poster session
Paper
Convergence to project success!

Slides | Video

Lab

Use this morning to finalize your group project. The 8:00-10:00 slot is for you to dedicate to project completion and the 10:00-12:00 lab session provides the last chance for in-person feedback on the project.

Lab

The information and material about the marked exercises is made available in this section. 

The marked exercises must be completed individually.

[Submission 1 Questions]

This section contains information and material about the Group Mini Project.

Theme: Deep learning to foster safer online spaces

Scope: The group mini-project aims to support a safer online environment by tackling hate speech in various forms, ranging from text and images to memes, videos, and audio content.

Objective: To develop deep learning models that help foster healthier online interactions by automatically identifying hate speech across diverse content formats. These deep learning models shall be carefully designed to prioritize accuracy and context comprehension, ensuring they differentiate between harmful hate speech and legitimate critical discourse or satire. 

Context: Developing deep learning models that help prevent the surfacing of hateful rhetoric will lead to a more respectful online environment where diverse voices can coexist and thrive.

Definition: What is hate speech?

Datasets: Initial list (OLID dataset license is being clarified)

Data Planning and Guidelines

Human Research Ethics Committee

Paper Template: File (.zip). When working on your paper, you can use to Overleaf, a free online LaTeX editor that provides an easy way of creating and editing LaTeX files. Documentation on how to work with Overleaf is available here.

Poster Template: File (.svg), File (.pptx)

Environment (cluster access tutorial): File (.zip) (updated 'docker run -it'), Video 

Two set of exercises submission + One mini group project submission