Deep RNN by Hand ✍️
Calculating AI by Hand: 9 of 28
Library › Calculating AI by Hand ✍️
Deep RNN by Hand ✍️
Reinforcement Learning with Human Feedback (RLHF) by Hand ✍️
A Deep Recurrent Neural Network (RNN) extends a basic single-layer RNN into multiple layers of hidden states, effectively incorporating deep learning into the RNN architecture.
How does a Deep RNN work?
Setup
Step 1 of 12: Given
A sequence of four inputs X1, X2, X3, X4 ⬛️
Recurrent weights and biases for hidden layers a 🟩, b 🟧, c 🟪, and the output layer y 🟦.
Step 2 of 12: Initialize Hidden States
Set a0, b0, c0 to zeros
Process X1 (t = 1)
Step 3 of 12: First Hidden Layer (a)
The transformation matrix is horizontal concatenation of input weights, hidden state weights and biases, visualized as [⬛️ | 🟩 | ⬜️] .
The state matrix is vertical concatenation of input X1, previous hidden state a0, and an extra 1, visualized as [⬛️ ; 🟩 ; 1].
Multiply the two matrices to obtain new hidden state a1 = [0 ; 1].