Time: W 9-10:30am
Location: 320 Soda, 1-2 units
Instructor: John Canny

Class Piazza is open

This course will cover systems and AI topics related to machine learning at very large scale. The course will cover deep learning and reinforcement learning as well as general machine learning models. We are particularly interested in deep learning and reinforcement learning on massive emerging datasets like camera data from computer assisted vehicles, drones etc. Single-machine training times on million-image datasets (ImageNet) are in the days. Effective parallelization methods have reduced that time to hours. But a single vehicle generates more than a million images each day, and we will soon be dealing with datasets millions or billions of times larger. This will require radically new approaches to training. Careful selection of training examples will be essential. "Learning to learn" inspired by human meta-cognition is a potential guiding principle.

The course will have one 1.5-hour meeting per week (Weds), and is available for one or two units of credit. For two unit credit, students will be expected to do a class project and presentation. First meeting Weds 1/20.

Date	Topic	Readings+Slides
1/20/2016	Background on machine performance: Memory, CPU, storage, network: roofline design	Lec 1 readings and slides
1/27/2016	Communication topologies. Systems: Hadoop, Spark, Graphlab, MPI.	Lec 2 readings and slides
2/3/2016	Systems: Parameter Servers, application to DNNs.	Lec 3 readings and slides
2/10/2016	Gradient Optimization: ADAGRAD, ADAM, Batch Normalization	Lec 4 readings and slides
2/17/2016	Monte-Carlo Optimization	Lec 5 readings and slides
2/24/2016	Project Plan Presentations	Project Plan
3/2/2016	Accelerating DNNs at scale: Elastic SGD	Lec 6 readings and slides
3/9/2016	Learning without Iteration: Tensor Factorization and DNNs	Lec 7 readings and slides
3/16/2016	Reinforcement Learning	Lec 8 readings and slides
3/30/2016	Project Updates	Project Updates
4/6/2016	Scaling up Reinforcement Learning	Lec 9 readings and slides
4/13/2016	Sample selection	Lec 10 readings and slides
4/20/2016	Learning to Learn	Lec 11 readings and slides
4/27/2016	Project Presentations	Project Presentations