Understanding the maths behind forward and back propagation is not very easy.<\/p>\n
There are some very good \u2013 but also very technical explanations.<\/p>\n
For example : The Matrix Calculus You Need For Deep Learning Terence Parr and Jeremy Howard\u00a0<\/a>is an excellent resource but still too complex for beginners.\u00a0<\/p>\n I found a much simpler explanation in the ml cheatsheet<\/a>.<\/p>\n The section below is based on this source.<\/p>\n I have tried to simplify this explanation as below.<\/p>\n All diagrams and equations are based on this source in the ml cheatsheet<\/a><\/p>\n \u00a0<\/p>\n \u00a0<\/p>\n \u00a0 \u00a0<\/p>\n Let\u2019s start with forward propagation<\/p>\n \u00a0<\/p>\n Here, input data is \u201cforward propagated\u201d through the network layer by layer to the final layer which outputs a prediction. The simple network can be seen as a series of nested functions. \u00a0<\/p>\n For the neural network above, a single pass of forward propagation translates mathematically to:<\/p>\n<\/p>\n A ( A( X Wh) Wo )<\/strong><\/p>\n \u00a0<\/p>\n Where\u00a0<\/p>\n A\u00a0is an activation function like\u00a0ReLU<\/a>,\u00a0<\/p>\n X\u00a0is the input<\/p>\n Wh and\u00a0Wo are weights for the hidden layer and output layer respectively<\/p>\n \u00a0<\/p>\n A more complex network can be shown as below<\/p>\n INPUT_LAYER_SIZE = 1<\/p>\n HIDDEN_LAYER_SIZE = 2<\/p>\n OUTPUT_LAYER_SIZE = 2<\/p>\n \u00a0<\/p>\n In matrix form, this is represented as:<\/p>\n \u00a0<\/p>\n \u00a0 \u00a0<\/p>\n \u00a0<\/p>\n \u00a0<\/p>\n \u00a0<\/p>\nForward propagation<\/h2>\n
<\/a><\/p>\n
<\/a>\u00a0<\/p>\n
<\/a><\/p>\n