Course Content
Basics of pattern recognition, Neural networks. Introduction to deep learning, convolutional networks, Applications in audio, image and text. All coding done in Python.Contact Details
| Instructor | Sriram Ganapathy |
| Office | Electrical Engineering C 334 (2nd Floor) |
| deeplearning doT cce2020 aT gmail doT com | |
| Teaching Assistants | Prachi Singh (Electrical Engineering C 328 (2nd Floor)) |
Textbooks
- “Pattern Recognition and Machine Learning”, C.M. Bishop, 2nd Edition, Springer, 2007.
- “Neural Networks”, C.M. Bishop, 2nd Edition, Springer, 1995.
- “Deep Learning”, I. Goodfellow, Y, Bengio, A. Courville, MIT Press, 2016. html
Pre-requisites
- Random Process/Probablity and Statistics - Recommended Book "Probability and Statistics" by Stark and Woods. Video Lectures
- Linear Algebra/Matrix Theory - Recommended book "Linear Algebra", by G. Strang - Author's webpage for Videos and Text
- Basic Programming with Python - numpy based array and matrix operations.
- Calculus
Announcements
- Final class on May 14,2020
- Final Exam on June 7, 2020 from 6pm. Open book and notes. Online exam, question paper sent through email and answers returned over email before deadline.
- Google group for discussions and announcements: dl_cce2020
Slides
| 16-01-2020 | Introduction to Deep Learning Course. Examples. Roadmap of the course. |
slides |
code |
| 23-01-2020 | Decision Theory for Machine Learning. Maximum Posterior Probability Rule. Minimum mean square estimation for Regression. |
slides | |
| 6-2-2020 | Matrix Calculus. Differentiating vectors and matrices. Principal Component Analysis. Preprocessing data - standardizing and whitening. |
slides | |
| 13-2-2020 | Linear Regression. Regularization. Least squares model for classification. Logistic regression Lecture-1. |
slides | |
| 20-2-2020 | Linear Regression. Regularization. Least squares model for classification. Logistic regression Lecture-2. |
slides | |
|
Second assignment due on March 10.
|
HW2.pdf | Preprocessing code | |
| 27-2-2020 | Linear Regression. Regularization. Least squares model for classification. Logistic regression Lecture-3. |
slides | |
| 5-3-2020 | Midterm Exam |
||
| 12-3-2020 | Neural networks with one or more hidden layers. Error function. Model update using gradient descent. Backpropogation algorithm. Stochastic gradient descent algorithm. |
slides | |
|
Third assignment due on April 17.
|
HW3.pdf | Preprocessing code | |
| 02-04-2020 | Discussion on Depth Versus Width. Practical considerations in Deep Learning. Avoiding Overfitting- Regularization, Dropout. Convolutional Neural Networks. |
slides
Video |
|
| 09-04-2020 | Dropout in detail. Training and Testing with Dropouts. Convolutional neural networks. Convolution, maxpooling operations. |
slides
Video |
|
| 16-04-2020 | CNNs in detail. DNN versus CNN in terms of number of parameters. Backpropogation in CNNs. Convolution and Maxpooling backpropagation. |
slides
Video |
|
| 23-04-2020 | t-SNE (t-distributed stochastic neighborhood embedding) for data visualization. Understanding deep networks using tSNE. Using image classification examples. Identifying representation learning using reconstruction. Deep networks in speech processing. |
slides
Video |
|
| 30-04-2020 | Recurrent Neural Networks. Forward and Backward propagation. Various Architectures for sequence to sequence and sequence to vector mapping. |
slides
Video |
|
| 07-05-2020 | Recap of recurrent networks. Architectures for vector to sequence mapping and encoder decoder sequences. Introduction to LSTM-RNNs. Attention mechanism in encoder decoder models |
slides
Video |
|
14-05-2020 |
Fourth assignment due on May 31. |
HW4.pdf | |
| 14-05-2020 | Recap of attention mechanism in encoder decoder models. Unsupervised models for deep learning. Autoencoders and generative adversarial networks. Image generation using GANs. |
slides
Video |