check out my IDL_notes.pdf lecture notes! if you'd rather not download the file, check the google drive link HERE, or either one of those linked-in discussions 1/2
Chapters recap:
| Chapter | Sections recap |
|---|---|
| Basic NN model | MLP, Activations, Loss function, Backpropagation |
| deep NN theory | Universal Approximation Thm., Shallow vs deep |
| LTI systems and Convolutional-NN | Time/Translation Invariance, Convolutional NN |
| Rerurrent-NN | Elman network, Backpropagation through time, LSTM, GRU |
| Attention Layers | Attention, self Attention, Multi-head Attention, Transformers |
| Auto-Encoders | Auto-Encoders, VAE, WAE |
| Generative Models | GAN, cGAN, GLOW, GLO |
| Optimizations | Sharp/Smooth minima, Momentum, AdaGrad, Adam |
| Exercise | Description |
|---|---|
| ex1 | implementing MLP to identify peptides from the Spike protein of the SARS-CoV-2 virus |
| ex2 | comparing Elman network (basic RNN), GRU cell, and MLP with restricted self-attention layer in classifying movie reviews as positive or negative |
| ex3 | using GAN and conditional GAN (cGAN) to generate novel samples of the MNIST dataset |