Gradient descent

Also called: SGD, stochastic gradient descent, optimizer, optimization

The method used to train neural networks: measure how wrong the model is, work out which direction to nudge each weight to be less wrong, take a small step and repeat millions of times.

For example: Like walking downhill in fog by always stepping where the ground slopes down, gradient descent moves the weights toward lower error.