Machine learning II

The course takes place on Wednesday from 13:15 to 16:00 in the room INf 119.

========================

Format of the course:

The course format is that of a split-class.

• 45 minutes long video that presents the theory of the course must be watched prior to coming to class.  
• 2 x 45 = 1h30 minutes of interactive lecture and interactive exercise session. 
• 45 minutes for programming session, alternating with Q&A session on projects (look at schedule).

The interactive lecture will be given in class. If infrastructure allows, we will also give it on zoom for those who prefer to attend remotely at: https://epfl.zoom.us/j/62262147163

The interactive exercise session and programming session will be given on site.

========================

A repository of all lecture videos is available at: https://mediaspace.epfl.ch/channel/LASA+-+Machine+Learning+Courses/30562

========================

Grading:  40% of the Grade will be based on personal work done during the semester (this entails either doing a computer based mini-project (code competition) or presenting papers an advanced topic in the class debates, see Practicals section below). The remaining 60% of the grade will be based on a 30 minutes oral exam (15 minutes preparation, 15 minutes oral defense). The oral will examine the student on the material viewed during the course.

You must choose between participating in the coding competition or in a debate to earn the 40% of the grade based on personal work. 

Debates: A set of topics for the panels/debates will be announced by the instructor. You can choose which camp you will defend. Debates last 30 minutes and take place in class. To prepare for the panel/debate, you must read associated literature and prepare a deck of slides and a 2 pages summary of the main arguments you plan to bring to the table. The panels/debates unfolds as follows: first each debater starts by presenting their position in 1-2 slides, then they debate with one another and with the audience for 15-20 minutes. During the live debate, debaters must back their arguments by using some of their additional slides that contain examples or more details facts. The audience votes on which side wins! Grade is based on the quality of the content of the arguments,  slides and summary. Grade is not influenced by the outcome of the audience's vote!

Code Competitions :  Several datasets are offered - this an unsupervised learning problem - the goal is to extract insightful information from the data, such as inferring patterns, identifying features that are instrumental and features that are irrelevant, extracting outliers, etc.  To extract this information, you need to use one or more algorithms of your choice,. You should modulate the algorithm to obtain best performance, knowing algorithm's sensitivity to parameters' choices. You report on the results you have found in a 2-page summary, and a set of slides which you present in class. The most insightful analysis wins! Grade is based on the quality of the summary, slides and presentation. Grade is not influenced by the outcome of the competition!

Instructions and topics are available HERE.

You can sign up for debates and code competitions using the two polls, see polls at the end of this section.  

Deadline to register for a project: March 5 2025

Deadline to submit your files:

• Debates and Coding Competition: 2-4 pages summary and slides must be submitted by 13h00 (1pm) on Monday, May 26. 

Late submissions incur 1pt penalty per day late!  Summary and slides must be uploaded on moodle, see link at the bottom of this section.

Schedule for Oral presentation of Competition and Debates will be posted in due time.

========================

Online Resources in Machine Learning:

• Online repository of papers from NIPS conference
• Pascal: Network of Excellence on Pattern Recognition, Statistical and Computational Learning (summer schools and workshops)
• ML List: "Archives of the Machine Learning List"
  
• ML Repository: "This is a repository of databases, domain theories and data generators that are used by the machine learning community for the empirical analysis of machine learning algorithms."
  
• Clever Methods for Overfitting

Recommended Textbooks:

General Introduction to Machine Learning:

• Machine Learning: A probabilistic perspective by Kevin P. Murphy, MIT Press
  
• Introduction to machine learning by A. Smola and S.V.N. Vishwanathan, on-line version
  
• "Bayesian Reasoning and Machine Learning" by D. Barber, Cambridge University Press

Kernel Methods: PCA, SVM:

• "Kernel Methods for Pattern Analysis" by John Shawe-Taylor, Nello Cristianini, Cambridge University Press (June 28, 2004)
• "Pattern Recognition and Machine Learning" by Christopher M. Bishop, Springer; 1 edition (October 1, 2007)
• Learning with Kernels by B. Scholkopf and A. Smola, MIT Press 2002
  

Statistical Learning Methods:

• "Pattern Classification" by Richard O. Duda, Peter E. Hart, David G. Stork, Wiley-Interscience; 2 Sub edition (October 2000)
• Introduction to Statistical Learning Theory by Olivier Bousquet, Stephane Boucheron, and Gabor Lugosi
• Information Theory, Inference and Learning Algorithms by David J.C Mackay, Cambridge University Press, 2003
  

Neural Networks:

• Spiking Neuron Models by W. Gerstner and W. M. Kistler, Cambridge University Press, Aug. 2002
  
• "Hebbian Learning and Negative Feedback Networks (Advanced Information and Knowledge Processing)" by C. Fyfe, Springer
• "Independent Component Analysis", A. Hyvarinen, J. Karhunen and E. Oja, Wiley Inter-Sciences, 2001
  
• "Self-Organizing Maps", Teuvo Kohonen, Springer Series in Information Sciences, 30, Springer, 2001
• "Introduction to Neural Networks: A Comprehensive Foundation" (2nd Edition) by S. Haykins

Reinforcement Learning:

• "Reinforcement Learning: An Introduction", R. Sutton & A. Barto, A Bradford Book. MIT Press, 1998
  
• "Reinforcement Learning: A Survey", Leslie Pack Kaelbling & Michael L. Littman and Andrew W. Moore, Journal of Artificial Intelligence Research, Volume 4, 1996
  

Course materials: Course materials are divided into several categories defined as follows

• L : Lecture slides and Solutions to exercises
• LX : Supplementary lectures (extra topics not evaluated)
• TP : Practicals-related materials (Description, Matlab code, and Solutions)
• MP : Mini-projects related information
• RD : Related Documentation (Textbook excerpt, etc.).
• AM : Additional materials (Matlab code, etc.)

========================

Lecture Notes:

The Lecture Notes of this course can be downloaded by clicking on the link below:

Lecture

• Introduction to class format
  
• Brief overview of topics we will see in class

Practice session

• Introduction to coding projects and literature reading for debates
  - Registration poll for the coding project: link
  - Registration poll for the debates project: link

TODO prior to coming to class:
Interactive Lecture and Exercises (1h15-3pm)

• Kernels continued
• Kernel-PCA (kPCA)

Interactive Lecture & Exercises - 1:15-3pm

• kernel CCA
  

Practice session Coding Competition: 3:15-4pm

• kernel K-means
  

Q&A session on Coding Competition & Debates

• How to choose and analyse an algorithm
• How to search for literature.

TA in charge of solution: Yongtao

This class is only a practice session. Bring your own laptop and exercise your understanding of kernal PCA, CCA and kernal K-means

• Spectral Clustering & non-linear embedding

TA in class: Yongtao & Sthit

Objective:

Practice-Session: This is a 3-hour practice session on manifold learning methods. Come in class and do practice session 2 on matlab using your laptop.

Coding Project: Use also this time to ask TA for help on your coding project.

• (Optional:) Video recal of SVM and weaknesses
• SVM Limitations
• Sparse SVM - nu-SVM and Relevance Vector Machine (RVM)
  
• SVM Semi-Supervised Clustering - Transductive SVM
• SVM clustering - SVC
  

TA:  Baiyu is present and presents the solution

Objectives:

Practice session: This is a practice session on sparse SVM and other classification methods seen in class

Coding project: Take the time to ask TA for help with your coding project

Support Vector Machine (SVM) and extensions

Wednesday April 1 Lecture 

• Lecture

Friday April 3: 

• Exercise session
  

• 1:15-2pm Interactive lecture on nonlinear regression
  3h15-4pm:  Q&A Debate and Coding Competition for GROUP 1 ONLY

This class is devoted to students presenting the results of their coding projector or debate discussion. The schedule is described below.

Beware that the 4-pages summary for debate and coding competitions are due on May 26, 13h00 (1pm)

Schedule for presentations:

13h15-13h30: Debate Success of ML - Sofiane Walid against TAs/Teacher + classroom

13h30-13h40: Coding competition Average Monthly Surface Temperature  Benjamin Beretz

13h40_13h50:  Coding competition Average Monthly Surface Temperature - Xinyi Han

13:50-14:00: Coding competition Average Monthly Surface Temperature Q&A 

14:00-14:10: Coding competition - Netflix - Gregoire Gimenez

14:10-14:20: Coding competition - Netflix - Irvin Dalaud

14:20-14:30: Coding competition Netflix Q&A 

14:30-14:40: Coding competition - Traffic Accident - Valentin Perret

14:40-14:50: Coding competition - Traffic Accident - Damien Vincent

14:50-15:00: Coding competition traffic Accident Q&A 

15:00-15:10: Coding competition - Food Nutrition - Quentin Rossier

15:10-15:20: Coding competition - Food Nutrition - Osman Ornek

15:20-15:30: Coding competition - Food Nutrition - Q&A 

15:30 - 16h00: Grade Deliberation - TA + Teacher (no students)

This class is devoted to students asking questions about the project preparation, GROUP 2 only.

This class is devoted to students presenting the results of their coding projector. The schedule is described below.

Students present in this order:

13h15-13h25: Coding competition - Calories Burned  - Maksymiliann Wojciech Schoeffel

13h25-13h35:  Coding competition - Calories Burned - Julien Ferdinand Gouraud

13h35-13h45:  Coding competition - Calories Burned - Q&A

13:45-13:55: Coding competition - AI/ML Salaries - Théo Pierre Luc Basseras

13:55-14:05: Coding competition - AI/ML Salaries - Thomas Jerver Asmussen

14:05-14:15: Coding competition - AI/ML Salaries - Q&A

14:15-14:25: Coding competition - Education & Career Success - Javier De Ramón Murillo

14:25-14:35: Coding competition - Education & Career Success - Rayan Bouchallouf

14:35-14:45: Coding competition - Education & Career Success - Q&A

14h45-14h55:  Coding competition - French employment, salaries, population - Mathieu Stawarz

14h55-15h05:  Coding competition - French employment, salaries, population - Marko Mitric

15h05-15h15:  Coding competition - French employment, salaries, population - Q&A

15:15 - 15h45: Grade Deliberation - TA + Teacher (no students)

Deadline for handing out report on mini-project and slides for oral presentation of papers: May 23, 12:00 (noon) . These must be submitted through moodle, see link below.