When  MW 3:45  5:15 pm S 8:30  10:00 am (Tentative) 
Where  EE B308 
Who  Sriram Ganapathy 
Office  C 334 (2nd Floor) 
sriramg aT iisc doT ac doT in  
Teaching Assistant  Akshara Soman 
Lab  C 328 (2nd Floor) 
aksharas aT iisc doT ac doT in 
Announcements
 Class Location will be B303 EE (Note the Change)
 Course Enrollment Form  Please fill up in case you interested in audit/credit here
 Fifth Assignment
 Posted here Due on 23112018.
 Second MidTerm Exam Date and Time Nov. 9 400 pm B308 Open Book, Open Notes.
 Project Midterm Review Nov. 17 10 am
 Presentation of 5 minutes for individual projects and 10minutes for group projects.
 2 slides on literature survey, 2 slides on progress thus far and 1 slide on future plan of the work.
Syllabus
 Introduction to real world signals  text, speech, image, video.
 Feature extraction and frontend signal processing  information rich representations, robustness to noise and artifacts.
 Basics of pattern recognition, Generative modeling  Gaussian and mixture Gaussian models, hidden Markov models.
 Discriminative modeling  support vector machines, neural networks and back propagation.
 Introduction to deep learning  convolutional and recurrent networks, pretraining and practical considerations in deep learning, understanding deep networks.
 Deep generative models  Autoencoders, Boltzmann machines, Adverserial Networks, Variational Learning.
 Applications in NLP, computer vision and speech recognition.
Grading Details
Assignments  15% 
Midterm exam.  20% 
Final exam.  35% 
Project  30% 
Prerequisites
 Must  Random Process/Probablity and Statistics
 Must  Linear Algebra/Matrix Theory
 Preferred  Basic Digital Signal Processing/Signals and Systems
Textbooks
 “Pattern Recognition and Machine Learning”, C.M. Bishop, 2nd Edition, Springer, 2011.
 “Neural Networks”, C.M. Bishop, Oxford Press, 1995.
 “Deep Learning”, I. Goodfellow, Y, Bengio, A. Courville, MIT Press, 2016. html
 “Fundamentals of speech recognition”, L. Rabiner and H. Juang, Prentice Hall, 1993.
References
 “Deep Learning : Methods and Applications”, Li Deng, Microsoft Technical Report.
 “Automatic Speech Recognition  Deep learning approach”  D. Yu, L. Deng, Springer, 2014.
 “Machine Learning for Audio, Image and Video Analysis”, F. Camastra, Vinciarelli, Springer, 2007. pdf
 Various Published Papers and Online Material
 Python Programming Basics pdf
Slides
06082018  Introduction to real world signals  text, speech, image, video. Learning as a pattern recognition problem. Examples. Roadmap of the course. 
slides 

08082018  Basics of Natural Language Processing  token, document and corpus. TFIDF features. Language modeling. Smoothing and backoff. Introduction to audio signal processing. DFT, STFT. Information Extraction Book (Chapter 6)  TFIDF Stanford Reading Material  Language Modeling 
Weblink Weblink 

13082018  Revisiting text processing. Perplexity. Shortterm Fourier Transform considerations. Timefrequency resolution. Melfrequency cepstral coefficient (MFCC) features. Image processing  filtering, convolutions. Matrix derivatives. Columbia Univ. STFT Tutorial PRML  Bishop (Appendix, Chapter 12) 
slides Weblink 

20082018  Unsupervised dimensionality reduction using Principal Component Analysis. Maximum variance formulation. Solution using eigenvectors of data covariance matrix. Minimum error formulation. Whitening and standardization. PCA for high dimensional data. PRML  Bishop (Chapter 12.1) 

27082018  Supervised dimensionality reduction using linear discriminant analysis (LDA). Fisher discriminant. Solution for 2 class LDA. Multiclass LDA. Comparison between PCA and LDA. Introduction to basics of decision theory. Inference and decision problems. Prior, likelihood and posterior. Maximumaposteriori decision rule for two class example. PRML  Bishop (Chapter 4.1.4) PRML  Bishop (Chapter 1.5) 
slides 

29082018  Decision theory for regression. MMSE estimation. Multivariate Gaussian Modeling. Intrepretation of Covariance. Diagonal and Full Covariance. Maximum Likelihood estimation of mean and covariance.
PRML  Bishop (Chapter 1.6) Further Reading 

30092018  Assignment #1. Due on 10092018. Analytical part submitted in class. Coding part submitted via mlsp18.iisc aT gmail doT com. 
HW1 
image data 
speech data 
31082018  Shortcomings of single Gaussian modeling. Introduction to mixture Gaussian modeling. Properties and parameters. Expectation Maximization algorithm  auxillary Function, proof of conververgence. Ref  Tutorial on GMMs Proof of EM algorithm 
slides 

10092018  Expectation Maximization Algorithm for GMMs. Initiatialization using Kmeans. Other Considerations in GMMs. GMM example for unsupervised clustering. Ref  EM algorithm for GMMs 
slides 

12092018  Limitations of GMM modeling for sequence data. Markov Chains. Hidden Markov Model (HMM) definition. Three Problems in HMM. Evaluating the likelihood using HMM (Problem 1), Complexity reduction using forward variable and backward variable. "Fundamentals of Speech Recog.", Rabiner and Juang (Chapter 6) Ref  Rabiner Tutorial on HMMs 
Rabiner Slides on HMM  
13092018  Assignment #2. Analytical part submitted in class (26092018). Coding part submitted via mlsp18.iisc aT gmail doT com (28092018).. 
HW2 
speechmusicdata 

17092018  Assignment #1 Discussion. 

19092018  Infering the best state alignment  Viterbi algorithm for HMM (Solution to Prob. II). Training of HMM using EM algorithm  Baum Welch Algorithm (Solution to Prob. III). "Fundamentals of Speech Recog.", Rabiner and Juang (Chapter 6) Ref  Rabiner Tutorial on HMMs 

21092018  EM algorithm for HMM with GMM state distribution. Dealing with multiple observation sequences. Implementation issues in HMM. Applications of HMMs  action recognition, face emotion tracking "Fundamentals of Speech Recog.", Rabiner and Juang (Chapter 6) Ref  Video analysis with HMMs 

24092018  Probablistic PCA. Problem formulation  generative model of the data. EM algorithm for parameter estimation. Application of PPCA. PRML  Bishop (Chapter 12.2) 
slides 

01102018  Regularized linear Regression revisited  dual problem formulation. Gram Matrix. Kernel functions. Examples PRML  Bishop (Chapter 6) 

03102018  Midterm Exam  
10102018  Properties of kernel functions. Rules for constructing kernels. The RBF kernel. Maximum margin classifiers  problem formulation for linearly separable case. Optimization fundamentals  primal and dual problems, strong duality, KKT conditions. Application of KKT conditions to maximum margin classifiers. PRML  Bishop (Chapter 7) Introduction to convex optimization  Boyd (Chapter 5) Weblink to the book 

12102018  Assignment #3. Analytical part submitted in class and Coding part submitted via mlsp18.iisc aT gmail doT com (22102018).. 
HW3 

12102018  Maximum margin classfiers  overlapping class distribution. Slack variables. Primal and Dual formualation. KKT conditions. Applications of SVM for text classification, cancer detection, MNIST. PRML  Bishop (Chapter 7) 
slides SVMApplicationslides 

15102018  Support vector regression  primal and dual, KKT conditions. Introduction to artificial neural networks  extension of kernel machines. Perceptron model. Learning rule in perceptron. Multilayer perceptron PRML  Bishop (Chapter 7) NNPR  Bishop (Chapter 3,4) 

17102018  Forward pass in MLP. Backpropagation algorithm  recursion. Choice of hidden layer activation function. NNPR  Bishop (Chapter 4) 

22102018  Computational complexity in Gradient Descent. Definition of Jacobian and Hessian matrices. Approximation to Hessian matrix computation. Choice of error function. Mean square and conditional expectation. Conditional expectation for classification with onehotencoding. Neural networks estimate posterior probablities. NNPR  Bishop (Chapter 6) 

24102018  Assignment #3. Analytical part submitted in class and Coding part submitted via mlsp18.iisc aT gmail doT com (22102018).. 
HW4 

24102018  Cross entropy for two class. Expected cross entropy loss and posterior probability estimation. General condition on error function for outputs to be posterior probability. Weight learning  gradient descent method. Properties of gradient descent using quadratic approximation. Learning rate parameter. NNPR  Bishop (Chapter 6,7) 
slides 

29102018  Drawbacks of gradient descent. Momentum in learning. Nesterov Accelerated gradient. Second order learning methods. Approximate Hessian. Data preprocessing for Neural networks. Decomposing the error into bias and variance. NNPR  Bishop (Chapter 7,9) 

31102018  Improving generalization in deep learning. Regularization  weight decay. Impact of regularization on weight update. Early stopping. Training with noise. Committee of Neural networks. Need for deep architectures. NNPR  Bishop (Chapter 9) 
slides 

02112018  Convolutional Neural Networks, Computation of convolutions. Number of parameters. Advantages over deep neural networks. Pooling and subsampling. Backpropagation in convolution. Insights in deep convolutional networks. Deep Learning Book  Goodfellow et al. (Chapter 9) 
slides 

05112018  Recurrent Neural Networks. Backprogation in time. Problem of vanishing gradients. Long short term memory networks. Various RNN architectures and applications. Deep Learning Book  Goodfellow et al. (Chapter 10) 
slides 

09112018  Midterm Exam 2  
12112018  Deep generative modeling  Restricted Boltzmann Machines (RBMs). Conditional independence property. RBM parameter learning. Positive and negative phase of learning. Intuitions behind contrastive divergence algorithm Deep Learning Book  Goodfellow et al. (Chapter 18,19,20) 
slides 

14112018  Assignment #5. Analytical part and Coding part submitted via mlsp18.iisc aT gmail doT com (23112018).. 
HW5 

14112018  Autoencoders. Denoising AE. Variational autoencoders. Variational lower bound derivation. KL divergence derivation. Data generation with VAEs Kingma's paper link VAE Tutorial link 
slides 

19112018  Generative Adversarial Nets (GANs), Attention Networks. Summary of MLSP course. 
slides 
