Course Content
Basics of pattern recognition, Neural networks. Introduction to deep learning, convolutional networks, Applications in audio, image and text. All coding done in Python.Contact Details
Instructor  Sriram Ganapathy 
Office  Electrical Engineering C 334 (2nd Floor) 
deeplearning doT cce2019 aT gmail doT com  
Teaching Assistants  Shreyas R, Purvi A (Electrical Engineering C 328 (2nd Floor)) 
Textbooks
 “Pattern Recognition and Machine Learning”, C.M. Bishop, 2nd Edition, Springer, 2007.
 “Neural Networks”, C.M. Bishop, 2nd Edition, Springer, 1995.
 “Deep Learning”, I. Goodfellow, Y, Bengio, A. Courville, MIT Press, 2016. html
Prerequisites
 Random Process/Probablity and Statistics  Recommended Book "Probability and Statistics" by Stark and Woods. Video Lectures
 Linear Algebra/Matrix Theory  Recommended book "Linear Algebra", by G. Strang  Author's webpage for Videos and Text
 Basic Programming with Python  numpy based array and matrix operations.
 Calculus
Announcements
 Midterm will be on March 7, 2019 in class hours.
 No class on the election day (April 18, 2019).
 Final exam will be on May 2, 2019, 67:45 pm. Venue: Lecture Hall 10, Lecture Hall Complex, Center for Continuing Education (CCE), near Prakruti Cafe, Indian Institute of Science, Bangalore560012.
Slides
Code for CNN
10012019 
Introduction to Deep Learning Course. Examples. Roadmap of the course.  slides  
17012019  Basics of Machine Learning  Decision and Inference Problems, Joint probability and posterior probabilities. Likelihood and priors. Loss matrix. Rule of maximum posterior probability. Loss function for regression. 
slides  
31012019  Matrix Derivatives. Maximum Likelihood estimation and Gaussian Example. Linear Models for Classification. Perpendicular distance of a point from a surface. Logistic regression. Ref  PRML, Bishop, Chap. 3. Appendix C 
slides  
07022019 
Logistic regression two class motivation. Posterior probability, sigmoid function, properties. Maximum likelihood for two class logistic regression. Cross entropy error for $K$ classes. Logistic regression for K classes, softmax function. Nonconvex optimization (local and global minima), Gradient Descent  motivation and algorithm.
Ref  PRML, Bishop, Chap. 4, Sec. 4.2 
slides  
14022019 
Summary of previous lectures. Gradient descent for $K$ class logistic regression. Implementing logistic regression for MNIST dataset. Perceptron Model and motivation. Introduction to single hidden layer neural networks. Training and Validation data sets. Logistic Regression Code Discussion.
Ref  PRML, Bishop, Sec. 4.2 and NN, Bishop, Sec. 3.5, 7.5 
Code for Logistic Regression  slides 
15022019  First Assignment posted  Due on Feb 24, 2019.  Assignment1 (with submission instructions)  
21022019 
Perceptron and 1 Hidden Layer Neural Networks. Nonlinear separability with hidden layer network. Type of hidden layer and output layer activations  sigmoid, tanh, relu, softmax functions. Error functions in MLPs.
Ref  NN, Bishop. Sec. 3.5, Sec. 4.8 
slides  
28022019 
Backpropagation in multilayer deep neural networks. Universal approximation properties of single hidden layer networks. Need for depth. The tradeoff between depth and width of networks. Representation learning in DNNs. Hierachical data abstractions. Example in Images
Ref  "Deep Learning", I. Goodfellow. Chap. 6 https://arxiv.org/abs/1311.2901 
slides Code for Single Layer NN  
07032019  Midterm Exam posted  Due by 19:45 hrs (90 minutes).  DL2019_Midterm1 

14032018 
Convolutional neural networks. Kernels and convolutional operations. Maxpooling and subsampling. Backpropagation in CNN layers. CNN example for MNIST.
Ref  "Deep Learning", I. Goodfellow. Chap. 9 
slides  
18032019 
Second assignment due on March 30 (Can attempt CNN questions after next class).

HW2.pdf  
21032019 
Solutions to Midterm 1 Posted.

DL2019_Midterm1_Solutions  
21032018 
Back Propagation in CNNs. Choice of Kernels and convolutional operations. CNN example for MNIST. Midterm Exam Discussion.
Ref  "Deep Learning", I. Goodfellow. Chap. 9 

28032019 
Recurrent neural networks, back propagation in recurrent neural networks. Different recurrent architectures  teacher forcing networks, encoder/decoder networks, bidirectional networks.
Ref  "Deep Learning", I. Goodfellow. Chap. 10 
slides 

04042019 
Issue of Vanishing and Exploding Gradients, Long Short Term Memory Networks (LSTM), Attention Mechanism in Neural Networks
Ref  "Deep Learning", I. Goodfellow. Chap. 10 
slides 

11042019 
Advancted Topics, Network in Network, Convolutional LSTM networks, Unsupervised learning, autoencoders, adversarial learning
Ref  "Deep Learning", I. Goodfellow. Chap. 16 
slides 

25042019 
Applications of Deep learning for natural language processing, image processing and speech processing.

slides 

26042019 
Third assignment posted, due on May 8.

HW3.pdf 