| When | MW 4:30 - 6:00 PM |
| Where | Microsoft Teams (Link: link) |
| Who | Sriram Ganapathy |
| Office | C 334 (2nd Floor) |
| sriramg aT iisc doT ac doT in | |
| Teaching Assistants | Jaswanth Reddy K, Prachi Singh, Akshara Sonam |
| Lab | C 328 (2nd Floor) |
| {jaswanthk, prachisingh, aksharas} aT iisc doT ac doT in |
Announcements
- First class on October 5, 2020 3:30 PM.
- Guidelines for monthly projects: Kit. It contains files to help you with the format for your project submissions.
- Monthly Project 2 final submissions can be uploaded to this folder: Link
- Follow the same formats for report and presentations from Monthly Project 1
- Monthly Project 2 presentations dates: December 29, 30 (and 31)
- Upload the Monthly Project 3 abstract here: link
- Deadline for Monthly Project 3 abstract submissions: Jan 10, 2021.
- Monthly Project 3 presentations date: 1st week of Feb. (TBA)
- Final Exam will be on January 23, 2021 Afternoon. Same format as Mid-Term
Syllabus
- Visual and Time Series Modeling: Semantic Models, Recurrent neural models and LSTM models, Encoder-decoder models, Attention models.
- Representation Learning, Causality And Explainability: t-SNE visualization, Hierarchical Representation, semantic embeddings, gradient and perturbation analysis, Topics in Explainable learning, Structural causal models.
- Unsupervised Learning: Restricted Boltzmann Machines, Variational Autoencoders, Generative Adversarial Networks.
- New Architectures: Capsule networks, End-to-end models, Transformer Networks.
- Applications: Applications in in NLP, Speech, Image/Video domains in all modules.
Grading Details
| 3 monthly research projects from three different domains (Speech/Audio, Text, Images/Videos, Biomedical, Financial, Chemical/Physical Sciences/Mathematical Sciences) | 60% |
| Midterm exam | 10% |
| Final exam | 30% |
Pre-requisites
- Linear Algebra
- Random Process
- Basic Machine Learning/Pattern Recognition course
- Good background in Python programming.
References
- A significant portion of the material would come from research papers/tutorials in the domain.
- Lecture notes in pdf format.
- “Deep Learning”, I. Goodfellow, Y, Bengio, A. Courville, MIT Press, 2016. html
Slides
| 05-10-2020 | Introduction. Setting the stage for the course. |
slides | video | |
| 07-10-2020 | Recap of deep learning - Notations, model parameters, feed forward networks, learning rule with stochastic gradient descent, convolutional networks. Need for recurrence networks. Types of recurrence. |
slides | video | notes |
| Reading Assignments | An overview of GD optimization algorithms Layer Normalization Batch Normalization |
|||
| 12-10-2020 | Recurrent neural networks : Forwards and Backward pass. Gradient propagation. Backpropagation through time (BPTT) algorithm. Vanishing gradients in Recurrent networks. |
slides | video | notes |
| 14-10-2020 | Recap of RNNs, BackPropagation Through Time (BPTT), LSTMs |
slides | video | notes |
| 19-10-2020 | Recap of RNNs, LSTMs, Bidirectional RNNs, Encoder-Decoder Models, Attention Networks |
slides | video | notes |
| 21-10-2020 | Recap of Encoder-Decoder Models, Visualizing Attention, Multi-head Attention, Self-Attention, Transformers |
slides | video | notes |
| 28-10-2020 | Self and multi-head attention, issues in RNN/LSTM, Introduction to transformer networks, transformer-encoder in detail. |
slides | video | notes |
| Reading Assignments | Neural Machine Translation By Jointly Learning to align and translate Attention is all you need |
|||
| Interesting Blogs | Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention) The Illustrated Transformer Animated RNN, LSTM and GRU Introduction to Neural Machine Translation with GPUs Attention and Augmented Recurrent Neural Networks |
|||
| 31-10-2020 | Tutorial - 1: Regularization, Optimization and Pytorch basics. |
video | ||
| 02-11-2020 | Transformer models in detail - encoder, self-attention and positional encoding. |
slides | video | notes |
| 04-11-2020 | Transformer models in detail - encoder, self-attention and positional encoding. |
slides | video | notes |
| 09-11-2020 | tSNE. |
slides | video | notes |
| Reading Assignments | Visualizing Data using t-SNE | |||
| More Sources | Visualizing the Hidden Activity of Artificial Neural Networks | |||
| 11-11-2020 | Unsupervised representation learning, Boltzmann machine and restricted Boltzmann machine. Model parameters, conditional independence. Issues in RBM training |
slides | video | notes |
| Reading Material | Deep Generative Models | |||
| 18-11-2020 | Restricted Boltzmann machine training, approximating the negative phase with Gibbs sampling.
Gaussian Bernoulli RBM - definiton and properties. Deep belief networks (DBNs). |
slides | video | notes |
| Reading Assignment | Restricted Boltzmann Machines for Collaborative Filtering | |||
| 23-11-2020 | Restricted Boltzmann machine training, Deep belief networks (DBNs) for initialization and visualization, Data Generation using RBMs, Variational Autoencoders |
slides | video | notes |
| 25-11-2020 | Variational autoencoders (derivations of the loss functions). The Variational lower bound. Model assumptions and approximations. |
slides | video | notes |
| Reading Assignment | Auto-Encoding Variational Bayes | |||
| 02-12-2020 | Variational autoencoders examples. Generating data using VAEs. Introduction to generative adversarial networks. GANs - loss function |
slides | video | notes |
| 07-12-2020 | Introduction to generative adversarial networks, GANs - loss function, min-max Game, Deep Convolutional GANs, Conditional GANs, CycleGANs, |
slides | video | notes |
| 09-12-2020 | Explainable Deep Learning - Motivation, Understanding hierarchical representations in deep learning. Transfer learning and representations. |
slides | video | notes |
| 14-12-2020 | Explainable Deep Learning - t-SNE embeddings for visualization, Understanding Deep Networks, Representations. |
slides | video | notes |
| 16-12-2020 | Explainable Deep Learning - Architecture updates for interpretability, Improving CAM without compromising architecture, Relation between CAM and Grad-CAM, Using attention mechanism for explainability. |
slides | video | notes |
| 21-12-2020 | Causality, Causal modeling, Structural Causal Equations, Causal inference and deep learning, Pruning based analysis of neural networks, Adversarial examples |
slides | video | notes |
| 23-12-2020 | Causal inference, Pruning based approach to analyzing/compressing, Criterion in involved in identifying importance, Approximating gradients, Adversarial examples and learning, Explainability with distillation, LIME model |
slides | video | notes |
| 28-12-2020 | Knowledge distillation, Knowledge distillation for explainability, Local Interpretable Model Agnostic Representation, LIME model, Future Research Directions, Capsule networks |
slides | video | notes |
| 30-12-2020 | Future Research Directions, Problem with current deep learning networks, Capsule networks, Capsule vs Neurons, Routing algorithm, Understanding the capsule output |
slides | video | notes |
| 04-01-2021 | Capsule networks, From a layer of neurons to layer of capsules, Capsule network performance, Automatic Sign Language Detection Task, Comparing capsule networks with other architectures, Deep learning on graphs |
slides | video | notes |
| 06-01-2021 | Deep learning on graphs, Graph convolutional networks, Semi-supervised learning using GCN |
slides | video | notes |
| 13-01-2021 | Modeling uncertainty in deep learning, Bayesian Deep Learning (Basics), Introduction to Gaussian processes, Allowing for noise in the model, Dropout and its Bayesian Interpretation |
slides | video | notes |
| 15-01-2021 | Bayesian Deep Learning, Gaussian processes for Bayesian inference, Dropout and its Bayesian Interpretation, Obtaining the model uncertainity |
slides | video | notes |