Automatic speech processing
EE-554
This file is part of the content downloaded from Automatic speech processing.
The course will take place in Room PH H3 33
Students joining the course online can join through the following Zoom link
https://idiap-ch.zoom.us/j/2732524500
No password is needed.
Lab exercises:
- Python versions are recommended, where available, and are most actively maintained
- Octave exercises have been updated and confirmed to work with Octave version 6.4.0 in October 2023
- Matlab exercises are provided as is, but have not been updated in a long time
Suggested Text Books
L. R. Rabiner and B-H Juang. Fundamentals of Speech Recognition. Prentice Hall 1993
B. Gold, N. Morgan and D. Ellis. Speech and Audio Signal Processing: Processing and Perception of Speech and Music. Wiley 2011.
X. Huang, A. Acero and H-W Hon. Spoken Language Processing: A guide to theory, algorithm and system development. Prentice Hall, 2001.
L. R. Rabiner and R. Schafer. Theory and Applications Digital Speech Processing. Pearson. 2010
L. R. Rabiner and R. Schafer. Digitial Processing of Speech Signals. Prentice Hall, 1978.
P. Taylor. Text-to-Speech Synthesis. Cambridge University Press, 2011.
B. Schuller and A. Batliner, Computational Paralinguistics: Emotion, Affect and Personality in Speech and Language Processing.
- News forum (Forum)
- Discussion forum (Forum)
- Additional videos and lectures can be found on the ISCA SCOOT platform (URL)
- Questions sets (Folder)
- ASP Exam 2022-2023 (Folder)
- ASP Exam 2023-2024 (File)
- A Course in Phonetics by Peter Ladefoged and Keith Johnson (URL)
- ASP Exam 2024-2025 (File)
- Vote: Preferred Format for the Final Exam (Choice)
Week 1 (Sep 11, 2025)
- Introduction lecture slides (File)
- Lecture 1 recap questions (File)
- International Phonetics Alphabet (URL)
- Tones listening test (Folder)
- Critical band listening test (Folder)
- Decibel scale listening test (Folder)
- Equal loudness listening test (Folder)
- Introduction lecture recording - Sep 21, 2023 (URL)
- Introduction lecture Part 2 recording - Sep 28, 2023 (URL)
- INTRODUCTION LECTURE 2024-2025 RECORDING (Sep 12, 2024) (URL)
- INTRODUCTION LECTURE 2025-2026 (SEP 11, 2025) (URL)
Week 2 (Sep 18, 2025)
Speech Signal Processing part
- Speech signal analysis lecture slides (File)
- Lecture 2 - recap questions (File)
- Audacity Instructions (File)
- Speech signal processing exercise in Python (Jupyter notebook) (File)
- Speech signal processing exercise in MATLAB (File)
- Speech signal processing exercise in OCTAVE (updated) (File)
- Sampling and Quantization Lecture Recording (Oct 5, 2023) (URL)
- Time Domain Analysis Lecture Recording (October 5, 2023) (URL)
- SPEECH ACQUISTION and TIME DOMAIN ANALYSIS - WEEK 2 LECTURE -PART 1 (Sep 19, 2024) (URL)
- TIME DOMAIN ANALYSIS AND FREQUENCY DOMAIN ANALYSIS - WEEK 2 LECTURE - PART 2 (Sep 19, 2024) (URL)
- Basic phonetics, Sampling and Quantization, and Time Domain Analysis (SEP 18, 2025) (URL)
Week 3 (Sep 25, 2025)
Source-system decoding, speech coding and feature extraction.
- Source-system decomposition, Speech coding and Feature extraction lecture slides (File)
- From frequency to quefrency: a history of the cepstrum by Oppenheim and Schafer (URL)
- Linear Prediction - A tutorial review by John Makhoul (URL)
- Lecture 3 recap questions (File)
- Speech Signal Acquisition and Analysis part Take Away Questions (File)
- Freq. domain analysis and Source-system decomposition Lecture recording - Oct 12, 2023 (URL)
- SPEECH ANALYSIS EXERCISE and SOURCE-SYSTEM DECOMPOSITION - WEEK 3 LECTURE (Sep, 26, 2024) (URL)
- Time domain analysis, Frequency domain analysis and Source-system decomposition (SEP 25, 2025) (URL)
Week 4 (Oct 2, 2025)
Discussion of answers to take away questions.
- Speech signal acquisition and processing Take Away Q&A recording (Oct 19, 2023) (URL)
- LINEAR PREDICTION BASED SOURCE-SYSTEM DECOMPOSITION, SPEECH CODING AND FEATURE VECTOR REPRESENTATION - LECTURE 4 - OCT 3, 2024 (URL)
- SUMMARY SHEET FOR SPEECH SIGNAL ANALYSIS PART (File)
- Source-System Decomposition Recap, LP-based Speech Coding and Feature Vector Representation (OCT 2, 2025) (URL)
Week 5 (Oct 9, 2025)
Statistical pattern recognition basics.
- Machine learning for speech processing (File)
- Statistical pattern recognition exercise in MATLAB (File)
- Statistical pattern recognition exercise in OCTAVE (updated) (File)
- SIGNAL ANALYSIS SUMMARY AND MACHINE LEARNING FOR SPEECH PROCESSING LECTURE RECORDING (Oct 10, 2024) (URL)
- LECTURE RECORDING Machine Learning for Speech Processing (OCT 9, 2025) (URL)
Week 6-7 (Oct 16 - Oct 30, 2025)
This course presents an overview on feature/representation learning
- Machine learning for speech processing (copy) (File)
- HMM exercise in Python (Jupyter notebook) (File)
- HMM exercise in OCTAVE (updated) (File)
- HMM exercise in MATLAB (File)
- A tutorial on hidden Markov models and selected applications in speech recognition by Rabiner (URL)
- A Gentle Tutorial of the EM algorithm and its application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models by Jeff Bilmes (URL)
- Combining Probability Distributions: A Critique and an Annotated Bibliography by Genest and Zidek (URL)
- Combining multiple classifiers by averaging or by multiplying? by Tax et al. (URL)
- Feature/Representation Learning Overview (slides) (File)
- FEATURE/REPRESENTATION LEARNING OVERVIEW LECTURE RECORDING (Oct 17, 2024) (URL)
- Lecture Discrete Markov Models, Hidden Markov Models and Detection Problems (OCT 16, 2025) (URL)
- Self-Supervised Learning for Speech Representations (File)
- Self-Supervised Learning for Speech Representations (pptx version with audio) (File)
- Self-Supervised Learning for Speech Representations Recording (Oct 30, 2025) (URL)
Week 8-10 (Nov 7 - Nov 20, 2025) - Automatic Speech Recognition
- Automatic Speech Recognition - Part 1 (File)
- Hidden markov model based speech recognition (File)
- Additional slides illustrating HMM-based Automatic Speech Recognition (File)
- Dynamic programming hands-on (File)
- Minimum edit distance (slides from Dan Jurafsky) (File)
- Automatic speech recognition - take away questions (File)
- LECTURE RECORDING AUTOMATIC SPEECH RECOGNITION - Part 1 (Oct 31, 2024) (URL)
- LECTURE RECORDING AUTOMATIC SPEECH RECOGNITION - PART 2 (Nov 7, 2024) (URL)
- LECTURE RECORDING AUTOMATIC SPEECH RECOGNITION - Part 3 (Nov 14, 2024) (URL)
- Whisper ASR Exercise (URL)
- NOV 6, 2025 LECTURE RECORDING - Feature Engineering and Automatic Speech Recognition (URL)
- NOV 13, 2025 - INSTANCE-BASED ASR and HMM-based ASR (URL)
- Posterior-based Approach (File)
- LECTURE RECORDING Posterior-based Approach (Nov 20, 2025) (URL)
Week 10-11 (Nov 20 - Nov 27, 2025) Text-to-Speech Synthesis
- Text-to-speech synthesis Part 1 (File)
- Text-to-speech synthesis Part 2 (File)
- Neural TTS overview slides (File)
- Neural TTS overview slides (pptx with audio) (File)
- XTTS TTS Exercise (URL)
- LECTURE RECORDING TEXT-TO-SPEECH SYNTHESIS PART 1 (Nov 21, 2024) (URL)
- LECTURE RECORDING TEXT-TO-SPEECH SYNTHESIS PART 2 (Nov 28, 2024) (URL)
- LECTURE RECORDING Text-to-Speech Synthesis Part 1 (Nov 20, 2025) (URL)
- LECTURE RECORDING Text-to-Speech Synthesis Part 2 (NOV 27, 2025) (URL)
- TTS approaches comparison (File)
Week 12-13 (Dec 4-11, 2025) Automatic Speaker Recognition
- Overview of Automatic Speaker Recognition (slides) (File)
- Automatic speaker recognition - Key take away questions (File)
- Low-dimensional speech representation based on Factor Analysis and its applications by Dehak and Shum (URL)
- AUTOMATIC SPEAKER RECOGNITION LECTURE RECORDING (Dec 5, 2024) (URL)
- LECTURE RECORDING AUTOMATIC SPEAKER RECOGNITION (Dec 4, 2025) (URL)
- Short recap of speaker recognition part (Dec 11, 2025) (URL)
Week 13 (Dec 11, 2025) Paralinguistic Speech Processing
- ML for Speech Processing slides updated with Paralinguistic Speech Processing (File)
- Deep learning based Speech Emotion Recognition (SER) lecture slides (File)
- Emotion Recognition exercise in Python (Jupyter notebook) (File)
- INTRODUCTION TO PARALINGUISTIC SPEECH PROCESSING LECTURE RECORDING (Dec 12, 2024)() (URL)
- Paralinguistic Speech Processing - Practical Aspects Slides (File)
- Parlinguistic Speech Processing LECTURE RECORDING (DEC 11, 2025) (URL)
Week 14 (Dec 18, 2025) Question-Answering
- AUTOMATIC SPEAKER RECOGNITION and QUESTION-ANSWERING LECTURE RECORDING (Dec 19, 2024) (URL)
- Overview on neural-based speech emotion recognition + Question-Answering + Partially open question set release RECORDING (DEC 18, 2025) (URL)
- AUTOMATIC SPEECH PROCESSING EXAM SAMPLE QUESTION SET (DEC 18, 2025) (File)
2023-2024 Week 8 (Nov 9, 2023)
This lecture dealt with feature vector representation, statistical pattern recognition (Q&A) and sequence matching, starting with string matching using dynamic programming.
- Feature parametrization and statistical pattern recognition (Lecture recording Nov 9, 2023) (URL)
- Pattern recognition (Q&A) and introduction to sequence matching (lecture recording Nov 9, 2023) (URL)
- End-to-End Acoustic Modeling using Convolutional Neural Networks for HMM-based Automatic Speech Recognition by Palaz, Magimai-Doss and Collobert (File)
Previous Year Week 9 (Nov 16, 2023)
Hidden Markov model based speech recognition
2023-2024 Week 9 (Nov 16, 2023)
In this week, we dealt with sequence matching (string matching and matching two speech utterances through dynamic programming), automatic speech recognition problem formulation, and discrete Markov model, and modeling of the set of hypotheses using DMM.
- Introduction to Automatic Speech Recognition (File)
- Sequence matching - Nov 16, 2023 first part lecture recording (URL)
- ASR formulation, Discrete Markov model, and Language Modeling - Nov 16, 2023 second part lecture recording (URL)
2023-2024 Week 10-11 (Nov 23-Nov 30, 2023)
Continuation of lecture on automatic speech recognition
- Introduction to Automatic Speech Recognition (copy) (File)
- Hidden markov model based speech recognition (copy) (File)
- Additional slides illustrating HMM-based Automatic Speech Recognition (copy) (File)
- Language Modeling, Knowledge-based ASR and Instance-based ASR lecture (Nov 23, 2023) - Part 1 (URL)
- HMM-based ASR Introduction lecture (Nov 23, 2023) - Part 2 (URL)
- HMM ASR fundamentals and HMM ASR Training (Nov 30, 2023) - Part 3 (URL)
- Practical aspects of HMM-based ASR (Nov 30, 2023) - Part 4 (URL)
2023-2024 Week 12-13 (Dec 7-Dec14, 2023)
Text-to-speech synthesis
- ASR Question-Answering and Speech Synthesis Intro - lecture recording (Dec 7, 2023) (URL)
- Text-to-speech synthesis - Part 2 - Lecture recording (Dec 14, 2023)) (URL)
- Text-to-speech synthesis Recap Questions (File)
2023-2024 Week 14 (Dec 21, 2023)
a) Automatic speaker recognition
b) Question-answering
c) Release of exam question set
- Automatic Speaker Recognition Lecture Recording (Dec 21, 2023) (URL)
- Automatic Speech Processing Exam Sample Questions (Exam on Jan 20, 2024) (File)
- Wu et al.,"Spoofing and countermeasures for speaker verification: A survey", Speech Communication, 2015 (URL)
Week 13 (Dec 14, 2023)
(a) Automatic speaker recognition continued
(b) An overview of paralinguistic speech processing
Week 14 (Dec 21, 2023)
Question-Answering