Machine Learning for Signal Processing
E9 205 • January 2025
Announcements
Enrollment Form
MLSP25 Enrollment Form
Logistics
Instructor
Dr. Sriram Ganapathy
sriramg@iisc.ac.inOffice: C 334 (2nd Floor)
Hours: Thu 2 PM - 3 PM (C334)
Class Times
Mon & Wed
11:00 AM – 12:30 PM
Venue: B308: EE
First Class: Jan 6, 2025
Teaching Assistants
Viveka Salinamakki
Varada R.
Hours: Thu 12 PM - 1 PM (C328)
Syllabus
- •Introduction to real world signals - text, speech, image, video.
- •Feature extraction and dimensionality reduction - principal components, linear discriminants.
- •Decision theory for pattern recognition, ML and MAP methods, Bias-variance trade-off, model assessment, cross-validation, estimating generalization error.
- •Generative modeling and density estimation - Gaussian and mixture Gaussian models, kernel density estimators, hidden Markov models. Expectation Maximization.
- •Linear regression and kernel methods. Regularization methods.
- •Discriminative modeling - support vector machines, decision trees and random forest classifiers, bagging and boosting.
- •Neural networks: gradient descent optimization and back propagation, regularization in neural networks, dropout, normalization methods.
- •Introduction to deep learning - feedforward, convolutional and recurrent networks, practical considerations in deep learning.
- •Introduction to transformer models - self and cross attention, encoder and decoder architectures, autoregressive decoding.
Grading Details
15%
Assignments
20%
Midterm Exam
25%
Project
40%
Final Exam
Pre-requisites
Must - Random Process/Probablity and Statistics
Must - Linear Algebra/Matrix Theory
Must - Coding Skills in Python
Textbooks
B1
Pattern Recognition and Machine Learning
C.M. Bishop, 2nd Edition, Springer, 2011.
B2
Neural Networks
C.M. Bishop, Oxford Press, 1995.
B3
Machine Learning: A Probabilistic Perspective
K. Murphy, 2012.
B4
References
Deep Learning : Methods and Applications
Li Deng, Microsoft Technical Report.
Automatic Speech Recognition - Deep learning approach
D. Yu, L. Deng, Springer, 2014.
Various Published Papers and Online Material
Python Programming Basics
PDFSlides
Date
Topic
Slides
05-01-2025
Introduction to real world data - text, speech, image, video. Learning as a pattern recognition problem. Examples. Roadmap of the course.
13-01-2025
Minimum Error Formulation of PCA. PCA for high dimensional data. Supervised dimensionality reduction - Linear Discriminant analysis
15-01-2025
Linear discriminant analysis (LDA), PCA versus LDA. Decision theory, Gaussian modeling
29-01-2025
Linear Regression, Choice of Basis, Regularized Linear Regression, and Bias Varinace Tradeoff
24-03-2025
Attention, Transformers, Multihead, Transformer Encoder and Decoder, Unsupervised Learning, and LLMs
26-03-2025
Encoder Decoder Attention, Unsupervised Learning, Self-Supervised Learning, LLMs, and Graphical Models