Interactive explorations of neural network fundamentals — built from scratch, visualized in depth.
Trains a four-layer MLP to approximate x² and visualises the full convergence trajectory, then reframes the trained network as a key-value memory — showing how neurons store receptive regions as keys and gradient-weighted contributions as values.
A minimal model built from scratch uses attention to learn color-to-noun mappings, showing how scaled dot-product attention implements a differentiable soft lookup table — and what the model learns geometrically after training.
A single-layer transformer trained on next-character prediction in a cyclic pangram, walking through every architectural component — causal masking, residual connections, layer normalisation — with geometric visualisations of what the model learns.