In the last article, the network read tokens and wrote them. But which token it picked had no influence on what came next. How does a network read what it writes? Each output becomes the next step's input. A network that feeds on its own outputs is called autoregressive.
d=1, h=2, y=1, seq=5, one-to-many
We start with a one-to-many network, like the decoder of the Seq2Seq article, reading a single number at its first step. The first input, x₁ = 1, starts it. After that, every input is 0, dimmed, so each step runs on its state alone. The five outputs are 0.76, −0.43, −0.24, 0.58, and −0.28. In the download, change x₁, and all five outputs follow.
