Calculating AI by Hand: 3 of 28
z3 should be [3, 3, 7]
When Calculating ∂L / ∂z2 by multiplying ∂L / ∂a2 with 1 for positive values and 0 otherwise.
∂L / ∂a2 = [1,-1] and then ∂L/ ∂z2 should be [1,0] right? then it should be a different ∂L/∂w2
do we need to transpose ∂L / ∂W1 to get W1 ?
There might be a mistake? ∂L / ∂z1 should be [ 1 , -2, 2 , -1] ?
Looks like ∂L / ∂z2 = [ 1 , 0 ] and ∂L / ∂z1 = [ 1 , 0 , 2 , 0 ] instead because of ReLU, no?
It seems that ReLU is not applied to gradients. Instead, ReLU only works on activation of the output.
z3 should be [3, 3, 7]
When Calculating ∂L / ∂z2 by multiplying ∂L / ∂a2 with 1 for positive values and 0 otherwise.
∂L / ∂a2 = [1,-1] and then ∂L/ ∂z2 should be [1,0] right? then it should be a different ∂L/∂w2
do we need to transpose ∂L / ∂W1 to get W1 ?
There might be a mistake? ∂L / ∂z1 should be [ 1 , -2, 2 , -1] ?
Looks like ∂L / ∂z2 = [ 1 , 0 ] and ∂L / ∂z1 = [ 1 , 0 , 2 , 0 ] instead because of ReLU, no?
It seems that ReLU is not applied to gradients. Instead, ReLU only works on activation of the output.