Course Content
Basics of pattern recognition, Neural networks. Introduction to deep learning, convolutional networks, Applications in audio and image processing. All coding done in Python.Contact Details
Instructor  Sriram Ganapathy 
Office  Electrical Engineering C 334 (2nd Floor) 
deeplearning doT cce2018 aT gmail doT com 
Textbooks
 “Pattern Recognition and Machine Learning”, C.M. Bishop, 2nd Edition, Springer, 2007.
 “Neural Networks”, C.M. Bishop, 2nd Edition, Springer, 1995.
 “Deep Learning”, I. Goodfellow, Y, Bengio, A. Courville, MIT Press, 2016. html
Prerequisites
 Random Process/Probablity and Statistics  Recommended Book "Probability and Statistics" by Stark and Woods. Video Lectures
 Linear Algebra/Matrix Theory  Recommended book "Linear Algebra", by G. Strang  Author's webpage for Videos and Text
 Basic Programming with Python  numpy based array and matrix operations.
 Calculus
Slides
11012018 
Introduction to Deep Learning Course. Examples. Roadmap of the course.  slides  
18012018  Basics of Machine Learning  Decision and Inference Problems, Joint probability and posterior probabilities. Likelihood and priors. Loss matrix. Rule of maximum posterior probability. Loss function for regression. 
slides  
01022018  Matrix Derivatives. Maximum Likelihood estimation and Gaussian Example. Linear Models for Classification. Perpendicular distance of a point from a surface. Logistic regression. Ref  PRML, Bishop, Chap. 3. Appendix C 
slides  
08022018 
Logistic regression two class motivation. Posterior probability, sigmoid function, properties. Maximum likelihood for two class logistic regression. Cross entropy error for two class.
Ref  PRML, Bishop, Chap. 4, Sec. 4.2 
slides  
15022018  Logistic regression for K classes, softmax function. Nonconvex optimization (local and global minima), Gradient Descent  motivation and algorithm. Ref  PRML, Bishop, Sec. 4.2 and NN, Bishop, Sec. 7.5 
Code for Logistic Regression  
22022018  First Assignment posted  Due on March 04, 2018. 
HW1  
22022018 
Training and Validation data sets. Logistic Regression Code Discussion. Perceptron and 1 Hidden Layer Neural Networks. Nonlinear separability with hidden layer network.
Ref  NN, Bishop. Sec. 3.5 
slides  
01032018 
Multilayer perceptrons, type of hidden layer and output layer activations  sigmoid, tanh, relu, softmax functions. Error functions in MLPs. Backpropagation learning in MLP. MLP for logistic regression in Keras.
Ref  NN, Bishop. Sec. 4.8 
slides Code for MLP 

08032018  Midterm Exam in Class. 

15032018 
Backpropagation in multilayer deep neural networks. Universal approximation properties of single hidden layer networks. Need for depth. The tradeoff between depth and width of networks. Representation learning in DNNs. Hierachical data abstractions. Example in Images
Ref  "Deep Learning", I. Goodfellow. Chap. 6 https://arxiv.org/abs/1311.2901 
slides  
22032018 
Convolutional neural networks. Kernels and convolutional operations. Maxpooling and subsampling. Backpropagation in CNN layers. CNN example for MNIST.
Ref  "Deep Learning", I. Goodfellow. Chap. 9 
slides Code for CNN 

02042018 
Second assignment due on April 12.

HW2.pdf  
05042018 
Recurrent neural networks, back propagation in recurrent neural networks. Different recurrent architectures  teacher forcing networks, encoder/decoder networks, bidirectional networks.

slides 

12042018 
Vanishing gradient problem in RNNs. Long short term memory networks. Unsupervised representation learning  Restricted Boltzmann machines, Autoencoders. Discussion of midterm exam

slides 

26042018  Final Exam in Class at 6pm. 