Designing, Visualizing and Understanding Deep Neural Networks (Spring 2019)

CS182/282A Designing, Visualizing and Understanding Deep Neural Networks Spring 2019

Course Information

Less frequently-used course information has moved here.

Schedule

 Date           
Lecture Topic                                                       
 Reading                  
 Assignments/Section Notes                         
W 1/23 Introduction, Course Overview, Brief history of deep networks. Download pptx or Download pdf Introduction Links to an external site. from Deep Learning Links to an external site.
M 1/28 Machine learning concepts: Loss and Risk, Discriminative models, Linear and Logistic Regression. Download pptx or Download pdf Review: sections 1.1-1.3 and 6.6-6.9 from the Download CS189 book (skip KL-div). Do Python/Numpy tutorial Links to an external site. if needed. Section 1 notes Download pdf
W 1/30 Bias-Variance tradeoff, Regularization, SVMs, Multiclass classification, Softmax. Cross-validation. Download pptx and Download pdf sections 1.4-1.6, 6.10-6.11, from the Download CS189 book (skip Tikhonov) Assignment 1 out
M 2/4 Optimization, Stochastic Gradient Descent. Download pptx and Download pdf Chapter 8 Links to an external site. of Deep Learning Links to an external site.
Optimization Notes Links to an external site. 		Section 2 notes Download pdf
W 2/6 Backpropagation, Convolutional Networks. Download pptx and Download pdf
 		Backpropagation Notes Links to an external site.
Convnet notes Links to an external site.
M 2/11 CNN examples, Activation functions, Initialization. Download pptx and Download pdf Convnet notes Links to an external site.
Training Neural Networks 1 Links to an external site.
Training Neural Networks 2 Links to an external site. 		Project Proposal out Section 3 notes Download pdf
W 2/13 Training: Batch normalization, Dropout, Ensembles, Hyperparameter tuning. Download pptx and Download pdf Training Neural Networks 2 Links to an external site.
Training Neural Networks 3 Links to an external site.
M 2/18 Holiday: President's Day
Tu 2/19 Assignment 1 due 11pm
Assignment 2 out
W 2/20 Object Detection and Segmentation. Download pptx and Download pdf Introduction to Object Detection Links to an external site. (All sections except ParseNet, PSPNet, DeepLab, PANet and EncNet)   Introduction to Semantic Segmentation Methods Links to an external site. Project proposal due
Download Practice MT1
Older Download MT1
M 2/25 Recurrent Networks, LSTMs, applications. Download pptx and Download pdf RNN chapter Links to an external site. from Deep Learning Links to an external site.
Understanding LSTM Networks Links to an external site. 		Section 4 notes Download pdf
W 2/27 Visualizing Deep Networks. Download pptx and Download pdf Understanding Neural Networks Through Deep Visualization Links to an external site.
Feature Visualization Links to an external site. 		Download Practice Mid-term Solutions
M 3/4 Midterm 1
W 3/6 Semantic Models for Text. Download pptx and Download pdf WordToVec Links to an external site.
Skip-Thought Vectors Links to an external site. 		Project Checkpoint 1
due week of 3/11/19
M 3/11 Attention Networks. Download pptx and Download pdf Recurrent Models of Visual Attention Links to an external site. Assignment 2 due 11pm
Assignment 3 out
Section 6 Notes Download pdf
W 3/13 Natural Language Translation. Download pptx and Download pdf NMT by Jointly Learning to Align and Translate Links to an external site.
Attention is All You Need Links to an external site.
The Illustrated Transformer Links to an external site.
M 3/18 Text Question Answering. Download pptx and Download pdf End-to-End Memory Networks Links to an external site.
QANet Links to an external site. 		Section 7 Notes Download pdf
W 3/20 Neural Dialog Systems. Download pptx and Download pdf Learning End-to-End Goal Directed Dialog Links to an external site.
BERT: Pre-training of Deep Bidirectional Transformers Links to an external site.
3/25-29 Spring Break
M 4/1 Adversarial and Fooling Networks. Download pptx and Download pdf Adversarial Examples Links to an external site.
Transferability/blackbox attacks Links to an external site.
Physical perturbation Links to an external site. 		Download Practice Midterm 2
Section 8 Notes Download pdf
W 4/3 Generative Models Download pptx and Download pdf Variational Auto-Encoders Links to an external site.
Autoregressive Models Links to an external site.
Image Transformer Links to an external site. 		Assignment 3 due 11pm
Assignment 4 out
Th 4/4 MT2 review session
6-8pm in room 4 LeConte
M 4/8 Generative Adversarial Networks Download pptx Download pdf Generative Adversarial Networks Links to an external site.
DCGAN Paper Links to an external site.
Wasserstein GAN Blog Post Links to an external site. (From GAN to WGAN)		 Section 9 Notes Download pdf
W 4/10 Midterm 2
M 4/15 Imitation Learning Download pdf End-to-End Learning for Self-Driving Cars Links to an external site.
DAGGER Links to an external site.
Adversarial Imitation Learning Links to an external site. 		No Sections
W 4/17 Reinforcement Learning: Policy Gradients Download pdf Policy Gradient Methods, chapter 13 of "Reinforcement Learning" Links to an external site.
M 4/22 Reinforcement Learning:
Value-based methods pdf 		DQN paper (Nature) Links to an external site.
Asynchronous Methods for Deep RL Links to an external site.
Double DQN Links to an external site. 		Section 10 Notes Download pdf
W 4/24 Exploration Download pdf Count-Based Exploration and Intrinsic Motivation Links to an external site.
Curiosity-driven Exploration Links to an external site.
Infobot: Information Bottleneck for RL Links to an external site.
F 4/26 Assignment 4 due 11pm
M 4/29 Learning to Learn Download pdf
 		Learning to Learn blog post
Matching Networks for One-Shot Learning Links to an external site.
Model-Agnostic Meta-Learning Links to an external site. 		Section 11 Notes Download pdf
W 5/1 Playing Games Download pptx and Download pdf A Survey of Monte-Carlo Tree Search Methods Links to an external site. sections 1-3
Mastering the Game of Go without Human Knowledge Links to an external site.
Sa 5/4 Project Poster due 11pm
Tu 5/7 Final project poster session 2-4pm in 310 Jacobs Hall Project Poster
M 5/13 Project Report due 11pm

 

Public Domain This course content is offered under a Public Domain Links to an external site. license. Content in this course can be considered under this license unless otherwise noted.