Linear models

Lecturer: Victor Panaretos

Doctoral Assistant: Nikitas Georgakis

Student Assistant: Philipp Mayer

Coursebook entry.

Schedule

• Lectures: Friday, 13:15-15:00, room GCA330 (starting September 12).
  
• Exercises: Monday, 10:15-12:00, room ELD020 (starting September 15, no exercise session on September 8).
  
  

Schedule modifications in the last two weeks:

Slide update: the slide deck was extended on 5 December (additional material in slides 311-331 -- this additional material will not be examined)

Examination

Final Exam 12/1 (9.15-12.15)- Room allocation:

Surname A–F: CM 1 106
Surname G–Z: CM 1 120

The bonus applies as follows:

Non-rounded grade = 0.6*{Final Exam} + 0.4*max {Final Exam, Midterm}

(we will then round upward)

You will be allowed 1 handwritten (not printed from tablet/pdf/scan) A4 sheet (front and back) for the final.

For the midterm, you are allowed half that: 1 handwritten (not printed from tablet/pdf/scan) A4 side (only front).

Slides

Slides are available below as a .pdf file. The slides are also meant as lecture notes -- they are sufficiently detailed.

Video Lectures

You will find a complete set of video lectures below, recorded during the pandemic (but still largely compatible with the course material), including recommended viewing per week.

Q&A and Exercises

An exercise booklet containing all the exercises (can be modified/expanded during the semester) is available below. Solutions are provided in a separate file. We strongly recommend that you attempt the exercises in earnest and use the solutions only as a last resort. These exercises are complementary to theory covered during the lectures, and they are the main content of the exercise sessions. 

Note: the weekly breakdown represents recommended progress, but might not perfectly correspond precisely to the progress made in class.

Note: You can enter Ed via the following link https://edstem.org/eu/courses/2675/discussion

Problems recommended for next exercise class

11th session-8/12: Exercises 42,43,45 Exercise recommendation history and prognosis can be found below.
 

Practicals

Even though there will be no graded project and the exams will focus on the content of the lectures (complemented by the exercises), application of linear models to real data is crucial for genuine understanding of the methodology. We recommend using the R language for this purpose, see the R tutorials created by Leo Belzile. Not all the tutorials are relevant to this course, but they can be consulted as an extra resource. Also, sections 4.5.2 and 4.5.3 provide solutions to Practical 1 and 2, respectively.

Some data-oriented problems are provided below. We encourage you to solve the problems and seek out feedback from the TAs during the exercise sessions. The practicals will occasionally appear as part of the recommended progress.

The effort is to create videos organised by topic, so some lectures may be short of 90', while others longer than 90'. 

• Week 1 covers:

1. Introduction
2. Subspaces, Spectra, and Projections

• Week 2 covers:

1. NonNegative Definite and Covariance Matrices

• Week 3 covers:

1. Gaussian Random Vectors
2. Likelihood and Least Squares - up to 27'45"

• Week 4 covers:

1. Likelihood and Least Squares - from 27'45" onwards
2. Geometry and Least Squares
3. Distribution theory of Least Squares

• Week 5 covers: 

1. Assessing Significance and Fit
2. Optimality and Asymptotics - up to 30'31"

• Week 6 covers:

1. Optimality and Asymptotics - from 30'31" onwards
2. Regression Diagnostics - up to 01:11'01"

• Week 7 covers:

1. Regression Diagnostics - from 01:11'01" onwards
2. Nested Model Selection

• Week 8 covers:

1. Non-Nested Model Selection

• Week 9 covers:

1. Multicollinearity

• Week 10 covers:

1. Penalised Least Squares

• Week 11 covers:

1. Robust Regression

• Week 12 covers:

1. NonLinear Regression

• Week 13 covers:

1. NonParametric Regression

• Week 14 covers:

1. More on Splines

• after lecture 1 recommended exercises: 1-4
  
• after lectures 2-3 recommended exercises: 5-14, 46, 47, 48b, 49
• after lecture 4 recommended exercises: 15-20
  
• after lecture 5 recommended exercises: 21,22,24
• after lecture 6 recommended exercises: 23,25-28
• after lecture 7recommended exercises: 29-33
• after lecture 8 recommended exercises: 34-37
• after lecture 9 recommended exercises: 38-40
• after lecture 10 recommended exercises: 41,44,50
• after lecture 11 recommended exercises: 42,43,45
• after lecture 12 recommended exercises:
• after lecture 13 recommended exercises:

Errata will be posted here: (please feel free to communicate errata to the TAs)