Video details loaded
HomeMIT 18.065 Matrix Methods in Data Analysis, Signal Processing, and Machine Learning, Spring 2018Lecture 25: Stochastic Gradient Descent
Lecture 25: Stochastic Gradient Descent
53:03
Description
Professor Suvrit Sra gives this guest lecture on stochastic gradient descent (SGD), which randomly selects a minibatch of data at each step. The SGD is still the primary method for training large-scale machine learning systems.
SummaryFull gradient descent uses all data in each step.
Stochastic method uses a minibatch of data (often 1 sample!).
Each step is much faster and the descent starts well.
Later the points bounce around / time to stop!
This method is the favorite for weights in deep learning.
Related section in textbook: VI.5
Instructor: Prof. Suvrit Sra
