Linear models

Lecturer: Victor Panaretos
Assistants: J. Almond Stöcker and Mila Pfander

Coursebook entry.

Schedule

• Lectures: Friday, 13:15-15:00, room MA A3 30 (starting September 13).
          Exceptionally on 18/10 in room GC A3 30.
  
• Exercises: Monday, 10:15-12:00, room CM 12 21 (starting September 23, since 16/9 is a public holiday).
  
  

Examination

Midterm exam during semester: November 22 in the lecture slot.

Final exam on Thursday 30.01.2025 from 09h15 to 12h15 (CE 1 4).

Lecture (polycopié)

Slides are available below as a .pdf file.

Q&A and Exercises

An exercise booklet containing all the exercises (can be modified/expanded during the semester) is available below. Solutions are provided in a separate file. We strongly recommend that you attempt the exercises in earnest and use the solutions only as a last resort. These exercises are complementary to theory covered during the lectures, and they are the main content of the exercise sessions. 

Note: the weekly breakdown represents recommended progress, but might not perfectly correspond precisely to the progress made in class.

Problems recommended for next exercise class

Problem 41, 42 and 45 (Dec 16)
Exercise recommendation history and prognosis can be found below.
 

Practicals

Even though there will be no graded project and the exams will focus on the content of the lectures (complemented by the exercises), application of linear models to real data is crucial for genuine understanding of the methodology. We recommend using the R language for this purpose, see the R tutorials created by Leo Belzile. Not all the tutorials are relevant to this course, but they can be consulted as an extra resource. Also, sections 4.5.2 and 4.5.3 provide solutions to Practical 1 and 2, respectively.

Some data-oriented problems are provided below. We encourage you to solve the problems and seek out feedback from the TAs during the exercise sessions. The practicals will occasionally appear as part of the recommended progress.

The effort is to create videos organised by topic, so some lectures may be short of 90', while others longer than 90'. 

• Week 1 covers:

1. Introduction
2. Subspaces, Spectra, and Projections

• Week 2 covers:

1. NonNegative Definite and Covariance Matrices

• Week 3 covers:

1. Gaussian Random Vectors
2. Likelihood and Least Squares - up to 27'45"

• Week 4 covers:

1. Likelihood and Least Squares - from 27'45" onwards
2. Geometry and Least Squares
3. Distribution theory of Least Squares

• Week 5 covers: 

1. Assessing Significance and Fit
2. Optimality and Asymptotics - up to 30'31"

• Week 6 covers:

1. Optimality and Asymptotics - from 30'31" onwards
2. Regression Diagnostics - up to 01:11'01"

• Week 7 covers:

1. Regression Diagnostics - from 01:11'01" onwards
2. Nested Model Selection

• Week 8 covers:

1. Non-Nested Model Selection

• Week 9 covers:

1. Multicollinearity

• Week 10 covers:

1. Penalised Least Squares

• Week 11 covers:

1. Robust Regression

• Week 12 covers:

1. NonLinear Regression

• Week 13 covers:

1. NonParametric Regression

• Week 14 covers:

1. More on Splines

• after lecture 1: public holiday
  
• after lecture 2 recommended: 
  
  1. Problems 1,3,5
  2. Problems 46-47
• after lecture 3 recommended:
  
  1. Problem  4
  2. Problems 6-9
• after lecture 4 recommended:
  
  1. Problems 11,12
  2. Problem 13 and setting up R
• after lecture 5 recommended:
  

  1. Problems 14,15
  2. Problems 17,18
     
• after lecture 6 recommended:
  

  1. Problems 19-21
     
  2. Practical 1 if interested in further application/illustration
• after lecture 7 recommended:
  

  1. Problems 22-25
• after lecture 8 recommended:
  

  1. Problem 26 and 27
  2. Problem 29
• after lecture 9 recommended:
  
  1. Problem 28
  2. Problem 30 and 31
• after lecture 10 recommended:
  
  1. Practical 2
  2. Problem 33
• after lecture 11 recommended:
  
  1. Problems 34,35
  2. Practical 3
• after lecture 12 recommended:
  
  1. Problems 37, 38
  2. Problem 40
• after lecture 13 recommended:
  
  1. Problems 41, 42
  2. Problem 45
     

Errata will be posted here: (please feel free to communicate errata to the TAs)