Statistical signal and data processing through applications

Statistical Signal and Data Processing through Applications is the follow-up to Bachelor courses on signal processing, such as "Signal Processing for Communications", or the Master course “Signal Processing Foundations” where the basics of signal processing were introduced. Building up on the basic concepts of sampling, filtering and Fourier transforms, we address spectral estimation, signal detection, classification, and adaptive filtering, with an application oriented approach: We first introduce relevant modern applications, such as neurobiological data analysis, spread spectrum wireless communications, echo cancellation, and then discuss appropriate statistical methods and tools to tackle related problems. The idea is to develop a "toolbox" of signal and data processing methods and learn how to use it for:

• Solving interesting problems arising in attracting applications 
  
• Preparing signals and data to be fed to communication and machine learning algorithms
  

Content

1. Fundamentals of Statistical Signal Processing :

Signals and systems in the deterministic and stochastic case.

2. Methods and algorithms :

Parametric and non-parametric signal models (wide sense stationary, Gaussian, Markovian, auto regressive and white noise signals); Linear prediction and estimation (orthogonality principle and Wiener filter); Maximum likehood estimation and Bayesian a priori.

3. Statistical Signal Processing Tools for Spread Spectrum Wireless Transmissions :

Coding and decoding of information using position of pulses (annihilating filter approach); Avoiding interference with GPS (spectral mask and periodogram estimation); Spectrum estimation for classical radio transmissions (estimating frequencies of a harmonic signal).

4. Statistical Signal Processing Tools for the Analysis of Neurobiological Signals :

Identification of spikes (correlation-based methods); Characterization of multiple state neurons (Markovian models and maximum likelihood estimation); Classifying firing rates of neuron (Mixture models and the EM algorithm).

5. Statistical Signal Processing Tools for Echo Cancellation

Adaptively removing noise in communication or audio systems.

Lectures & Exercise Sessions

Lectures: Thursday, 14h15 - 17h00, BC04. Exercises: Friday, 11h15 - 13h00, BC04.
Homeworks can be found on this site, in the resource section (see below). Homework sessions will focus on what has previously been done in the course. The topics will be explained in more detail with practical examples and exercises which help in both understanding the subject and preparing for the final exam. Students will be given an opportunity to proactively shape the contents of these sessions, i.e. ask for additional explanations of a particular topic, more exam-related exercises etc.

Lecturer

Andrea Ridolfi
Telephone: +41 79 790 79 96
E-Mail: andrea.ridolfi@epfl.ch

Assistant

Salim Najib

E-Mail: salim.najib@epfl.ch

Evaluation

Midterm (20%), Mini-project (20%), Final exam (60%)

Please read carefully the section concerning the mini project!

Please notice that for the midterm and the final the only allowed document is a cheatsheet that will provided to you during the course.

Bibliography

The slides presented in class and distributed on Moodle completely cover the contents of the course. The following books provide further details on specific topics presented in class, as well as on related topic. 

Background books

• P. Prandoni, Signal Processing for Communications, EPFL Press
• A.V. Oppenheim, R.W. Schafer, Discrete Time Signal Processing, Prentice Hall, 1989
• B. Porat, A Course in Digital Signal Processing, John Wiley & Sons,1997
• A. Yger, Theorie et analyse du signal: Cours et initiation via Matlab et Scilab, Ellipses.
• C.T. Chen, Digital Signal Processing, Oxford University Press
• P. Thiran, Stochastic models for communication systems (ask the teachers for the password), lecture notes, 2005
• D. P. Bertsekas, J. N. Tsitsiklis, Introduction to Probability, Athena Scientific, 2002 (excellent book on probability)

More advanced books

• L. Debnath and P. Mikusinski, Introduction to Hilbert Spaces with Applications, Springer-Verlag, 1988.
• A.N. Shiryaev, Probability, Springer-Verlag, 2nd edition, 1996.
• S.M. Ross, Introduction to Probability Models, Third edition, 1985.
• P. Bremaud, An Introduction to Probabilistic Modeling, Springer-Verlag, 1988.
• P. Bremaud, Markov Chains, Springer-Verlag, 1999.
• P. Bremaud, Mathematical Principles of Signal Processing, Springer-Verlag, 2002.
• S.M. Ross, Stochastic Processes, John Wiley, 1983.
• B. Porat, Digital Processing of Random Signals, Prentice Hall,1994.
• P.M. Clarkson, Optimal and Adaptive Signal Processing, CRC Press, 1993.
• P. Stoïca and R. Moses, Introduction to Spectral Analysis, Prentice-Hall, 1997.
• M. Vetterli and J. Kovacevic, Wavelets and Subband Coding, Prentice-Hall, 1995.
• S. Mallat, A Wavelet Tour of Signal Processing, Elsevier, 2008.
• G. McLachlan, D. Peel, Finite Mixture Models, Wiley, 2000.

Week 1: Fundamentals of SSP

Week 2: Fundamentals of SSP & Models and Methods

Week 3: Models and Methods 

Week 4: Models and Methods

Week 5: Review

Week 6: SSDP tools for wireless transmissions

Week 7: SSDP tools for Neurobiological Spike Processing

Week 8: SSDP tools for neurobiological signal processing

Week 9: SSDP tools for neurobiological signal processing

Holidays!!!! Easter break.

Week 10: SSDP tools for neurobiological signal processing

Week 11:  SSDP tools for Echo Cancellation

 

Week 14: Presentation of the Mini Projects (Friday May 30)

Each group will have a time slot of 15 minutes. The goal of the presentation is to:

• Present the theoretical aspect of the mini-project, i.e., give a mini lesson on the tool studied (using slides & the projector).
• Present a demo;
• Answer questions of the class (reserve up to 5 minutes fo that).
  

Before 14h, each group should submit

• The slides of the mini lesson.
• The files/code of the demo
• A user manual of the demo describing what it does and how to run it.

Week 12: Ascension Day

The goals of the mini project are:

• Implement one (or more) of the tools seen in class;
• Explore more advanced / specific tools related to the tools seen in class, via scientific literature and numerical implementation;
• Present the tools to the class with a demo and a performance comparison.  
  

Such goals are achieved via specific tasks:

• Work as a team (everyone must contribute and be aware of every detail of the accomplished work);
• Implement the assigned tool seen in class (Python or Matlab);
• Test it on simulated and real data (real data will be provided);
• Submit a report on the test of the tool on simulated and real data (Assignment #1);
  
• Explore other advanced tools, not presented in class, outperforming the assigned tool (start from the suggested literature, and pursue the research of information on additional papers & books);
• Submit a report on the advanced tools (Assignment #2);
• Implement the new tools (Python or Matlab);
• Prepare a demo (on simulated and real data) comparing the tools;
• Prepare about 5-10 slides to present to your colleagues the tools, their comparison, a demo, and your conclusions.
• Submit the demo (with instructions), the presentation, and a short report (min 6 pages, max 10 page) on the mini-project (Assignment #3);

Like in every Research or Research & Development project, you have access to what has been already done by your predecessors. You are expect to understand what your predecessor has done and improve it by bringing some "innovation" (better solution, better explanation, better demo).

You will be evaluated on these tasks (and assignment), on the quality of problem solutions, on the quality of your implementation, on the quality of the "innovation", on the quality of the presented demo & results.

Notice for the students re-taking the course

As well as re-taking the final exam, you are requested to re-do a mini-project. In addition, the mini-project have to be of different topic than the one you have previously done.