Recurrent attention for infinite context, recursive transformers, state space models. To stay at the frontier, you need to go back to understand RNNs. I used to teach RNNs to give students the founda…
RNNs by Hand, in Excel
From a small neuron to a small language model
