Training and Optimization

What happens between the first epoch and a usable model: the shape of the loss surface, gradients that vanish or explode, neurons that stop learning, and the optimiser, learning-rate and normalisation choices that decide whether training converges or wastes a week of GPU time.