By the time you finish this section, you’ll understand how backpropagation works - and perhaps more importantly, you’ll have a stronger understanding of how this algorithm is used to train neural networks from scratch. Inside this implementation, we’ll build an actual neural network and train it using the back propagation algorithm. Matt Mazur’s excellent concrete example (with actual worked numbers) that demonstrates how backpropagation works.Īs you can see, there are no shortage of backpropagation guides - instead of regurgitating and reiterating what has been said by others hundreds of times before, I’m going to take a different approach and do what makes PyImageSearch publications special:Ĭonstruct an intuitive, easy to follow implementation of the backpropagation algorithm using the Python language.Stanford’s cs231n exploration and analysis of backpropagation.The heavily mathematically motivated Chapter 2 - How the backpropagation algorithm works from Neural Networks and Deep Learning by Michael Nielsen.Andrew Ng’s discussion on backpropagation inside the Machine Learning course by Coursera.There are quite literally hundreds (if not thousands) of tutorials on backpropagation available today. The original incarnation of backpropagation was introduced back in the 1970s, but it wasn’t until the seminal 1988 paper, Learning representations by back-propagating errors by Rumelhart, Hinton, and Williams, were we able to devise a faster algorithm, more adept to training deeper networks.

Backpropagation in neural networks