Physics / Math
Neural descent
A neural network learns digits live, drawn as physics: the loss landscape in 3D, gradient descent as a ball rolling downhill, momentum as inertia and minibatch noise as temperature.
What happens
Training a neural network IS physics: the loss is a potential energy over weight space, and learning is a particle rolling downhill. The 3D surface is that landscape - a two-direction slice through 1210 dimensions, redrawn around the network as it moves - the glowing ball is the network itself, and the golden trail is the descent it just performed. The sliders are mechanics: learning rate is the step, momentum is inertia that coasts through ravines, and temperature turns descent into Langevin dynamics - heat it and the ball rattles forever above the minimum, exactly like a thermal particle in a well. Meanwhile the cube panel shows the digit it is reading right now. Two directions out of 1210: the dip you are looking at need not be the deepest one there is.
Equations used in the module
Equations y = softmax(W₂ σ(W₁ x + b₁) + b₂) L = −Σ log y_label (cross-entropy) W ← W − η ∇L + momentum (descent as a ball rolling)
