Gradient descent

2 stories tagged Gradient descent on Learning Fanatic.

Lesson 1.3: Tensors, gradients and autograd by hand Build a small automatic-differentiation engine in Python, check its gradients against numerical estimates, and use it to fit a line by gradient descent.
Lesson 1.4: A first model: the bigram language model Train a bigram language model by gradient descent, check that it learns the probabilities a table of counts gives, and see what a one-word memory cannot do.