Autograd
How PyTorch computes gradients automatically via the chain rule.
Set requires_grad=True on a tensor and PyTorch starts recording every
operation performed on it into a graph. Calling .backward() on a final
scalar (like a ) runs
: it walks that graph backward,
writing a into .grad for every tensor that
needed one.
⚠️ Watch out: Gradients ACCUMULATE across multiple .backward() calls instead of resetting. Call w.grad.zero_() (or optimizer.zero_grad() once you're using one) before the next backward pass, or gradients from old steps silently leak into the new one.
Now practice it
Exercises that use what this lesson just covered.