Lecture 4
Adaptive Subgradient Methods for Online Learning and Stochastic Optimization Links to an external site. John Duchi, Elad Hazan, Yoram Singer, Journal of Machine Learning Research 2011 - this is a long paper, we just need a summary of results and implications, not the proofs.
Slides Download Slides by Arturo Fernandez
Adam: A Method for Stochastic Optimization
Links to an external site. Diederik Kingma, Jimmy Ba, Arxiv:1412.6980, 2015 (version 8)
Slides Download Slides by Jaya Narasimhan