Course Content
Basics of pattern recognition, Neural networks. Introduction to deep learning, convolutional networks, Applications in audio and image processing. All coding done in Python.Contact Details
| Instructor | Sriram Ganapathy |
| Office | Electrical Engineering C 334 (2nd Floor) |
| deeplearning doT cce2018 aT gmail doT com |
Textbooks
- “Pattern Recognition and Machine Learning”, C.M. Bishop, 2nd Edition, Springer, 2007.
- “Neural Networks”, C.M. Bishop, 2nd Edition, Springer, 1995.
- “Deep Learning”, I. Goodfellow, Y, Bengio, A. Courville, MIT Press, 2016. html
Pre-requisites
- Random Process/Probablity and Statistics - Recommended Book "Probability and Statistics" by Stark and Woods. Video Lectures
- Linear Algebra/Matrix Theory - Recommended book "Linear Algebra", by G. Strang - Author's webpage for Videos and Text
- Basic Programming with Python - numpy based array and matrix operations.
- Calculus
Slides
11-01-2018 |
Introduction to Deep Learning Course. Examples. Roadmap of the course. | slides | |
| 18-01-2018 | Basics of Machine Learning - Decision and Inference Problems, Joint probability and posterior probabilities. Likelihood and priors. Loss matrix. Rule of maximum posterior probability. Loss function for regression. |
slides | |
| 01-02-2018 | Matrix Derivatives. Maximum Likelihood estimation and Gaussian Example. Linear Models for Classification. Perpendicular distance of a point from a surface. Logistic regression. Ref - PRML, Bishop, Chap. 3. Appendix C |
slides | |
| 08-02-2018 |
Logistic regression two class motivation. Posterior probability, sigmoid function, properties. Maximum likelihood for two class logistic regression. Cross entropy error for two class.
Ref - PRML, Bishop, Chap. 4, Sec. 4.2 |
slides | |
| 15-02-2018 | Logistic regression for K classes, softmax function. Non-convex optimization (local and global minima), Gradient Descent - motivation and algorithm. Ref - PRML, Bishop, Sec. 4.2 and NN, Bishop, Sec. 7.5 |
Code for Logistic Regression | |
| 22-02-2018 | First Assignment posted - Due on March 04, 2018. |
HW1 | |
| 22-02-2018 |
Training and Validation data sets. Logistic Regression Code Discussion. Perceptron and 1 Hidden Layer Neural Networks. Non-linear separability with hidden layer network.
Ref - NN, Bishop. Sec. 3.5 |
slides | |
| 01-03-2018 |
Multi-layer perceptrons, type of hidden layer and output layer activations - sigmoid, tanh, relu, softmax functions. Error functions in MLPs. Backpropagation learning in MLP. MLP for logistic regression in Keras.
Ref - NN, Bishop. Sec. 4.8 |
slides Code for MLP |
08-03-2018 | Midterm Exam in Class. |
| 15-03-2018 |
Backpropagation in multi-layer deep neural networks. Universal approximation properties of single hidden layer networks. Need for depth. The trade-off between depth and width of networks. Representation learning in DNNs. Hierachical data abstractions. Example in Images
Ref - "Deep Learning", I. Goodfellow. Chap. 6 https://arxiv.org/abs/1311.2901 |
slides | |
| 22-03-2018 |
Convolutional neural networks. Kernels and convolutional operations. Maxpooling and subsampling. Backpropagation in CNN layers. CNN example for MNIST.
Ref - "Deep Learning", I. Goodfellow. Chap. 9 |
slides Code for CNN |
|
| 02-04-2018 |
Second assignment due on April 12.
|
HW2.pdf | |
| 05-04-2018 |
Recurrent neural networks, back propagation in recurrent neural networks. Different recurrent architectures - teacher forcing networks, encoder/decoder networks, bidirectional networks.
|
slides |
|
| 12-04-2018 |
Vanishing gradient problem in RNNs. Long short term memory networks. Unsupervised representation learning - Restricted Boltzmann machines, Autoencoders. Discussion of mid-term exam
|
slides |
26-04-2018 | Final Exam in Class at 6pm. |