CS294-129 Designing, Visualizing and Understanding Deep Neural Networks

Deep Networks have revolutionized computer vision, speech recognition and language translation. They have growing impact in many areas of science and engineering. They also do not follow a closed set of theoretical principles. In Yann Lecun's words they require "an interplay between intuitive insights, theoretical modeling, practical implementations, empirical studies, and scientific analyses." This course attempts to cover that ground, and has three goals:

* Design principles and best practices: design motifs that work well in particular domains, structure optimization and parameter optimization.

* Visualizing deep networks. Exploring the training and use of deep networks with visualization tools.

* Understanding deep networks. Methods with formal guarantees: generative and adversarial models, tensor factorization.

This is a first offering of the class, and it draws heavily from the excellent Stanford course CS231n: Convolutional Neural Networks for Visual Recognition Links to an external site.. by Li, Karpathy and Johnson.

Projects: 

The public projects from this semester are up now.

Logistics

• Course Number: CS 294-129 Fall 2016, UC Berkeley
• CCN: 34652
• Instructor: John Canny
• GSIs:
  • Yang You: youyang@cs.berkeley.edu
  • Daniel Seita: seita@berkeley.edu
• Time: MW 1pm - 2:30pm
• Location: Soda 306
• Discussion: Join Piazza Links to an external site. for announcements and to ask questions about the course
• Office hours:
  • John Canny - M 2:30-3:30, in 637 Soda
  • Yang You - WF 9:00-10:00AM in 576 Soda
  • Daniel Seita - Thurs 5:00-6:00PM in Visual Computing Lab (5th floor Soda Hall)

Lectures Online

Lectures will be live streamed at this link Links to an external site..

After class, they will be archived here Links to an external site..

Prerequisites

The prerequisites for this course are:

* Knowledge of calculus and linear algebra, Math 53/54 or equivalent. You'll need this throughout the course.

* Probability and Statistics, CS70 or Stat 134. We'll talk about continuous and discrete probability distributions. CS70 is bare minimum preparation, a stat course is better.

* Machine Learning, CS189. You can probably handle this course without a machine learning course before, but it will be very helpful.

* Programming, CS61B or equivalent. Assignments will mostly use Python. If you need some help, try this tutorial from CS231n Links to an external site.

Texts

 We'll frequently use the online book: Deep Learning Links to an external site. by Ian Goodfellow and Yoshua Bengio and Aaron Courville

Grading

• Class Participation: 10%
• Midterm: 20%
• Final Project (in groups): 40%
• Assignments : 30%

Slip Days

You can use up to 5 slip days for late assignments.

Schedule