Exercise: Backpropagation Shapes and Gradients
Prerequisites: Neural Networks from Scratch
Problem
Why does the derivative of
tanh appear as 1 - tanh(z)**2 in backpropagation?Hint
Start by naming the invariant, validation split, or numerical guarantee involved. Jump to the solution when you're ready.