Sampling a Sentence by Hand ✍️
Calculating AI by Hand: 16 of 28
Library › Calculating AI by Hand ✍️
Sampling a Sentence by Hand ✍️
Reinforcement Learning with Human Feedback (RLHF) by Hand ✍️
In my course, I go over how exactly each pixel in an image or each word in a sentence is sampled from a probability distribution. The most common visualization of a probability distribution of images or sentences is a cloud. But some students found the cloud representation still too "cloudy." Thus, I made this by-hand exercise, using concrete numbers to dispel any lingering cloud in their understanding.
Setup
Step 1 of 11: Given
Given a sequence of input embeddings, represented as yellow column vectors.
Step 2 of 11: Feed to LLM
Feed the input embeddings to the LLM.
First Word
Step 3 of 11: Predict Distribution
The LLM predicts the probability distribution of the next output word over a vocabulary of 10 words, which is represented as a column vector shaded in blue. Then, we calculate the cumulative distribution by hand and write the results in blue. To do so, we start with zero at the bottom, add the number in each cell as we go up, until we reach 100 at the top. Note that for a valid probability distribution, they should add up to 100 (percentage).