Understanding Transformers · Part 10 of 16

Logits and Softmax

How the model turns a final vector into a probability for every possible next token — and why the word “logit” shows up twice.

The intuition: raw scores before they're normalized

Picture a judge assigning private scores to contestants before anyone converts them into percentages. A score can be positive or negative, large or small; it needn't sit between 0 and 1, and the scores needn't add up to anything in particular. A logit is that kind of number — a raw preference score, not yet a probability.

For the prompt “The cat sat on the …”, the model might hold these logits over a few candidate tokens:

Softmax: from scores to a distribution

Softmax converts a list of logits into probabilities:

Exponentiating makes larger logits dominate; dividing by the total forces the results to sum to one. Softmax doesn't invent the model's preference — it expresses the preference already sitting in the logits as a proper distribution.

Where the vocabulary logits come from

After the final transformer layer, the current position's hidden vector is multiplied by an output matrix. If the hidden size is d and the vocabulary has V tokens, that matrix maps d numbers to V numbers — one logit per token in the whole vocabulary.

The output matrix WU, sometimes called the unembedding matrix, is often not a separate parameter at all: many models tie it to the transpose of the embedding table, so the same weights that map a token to its starting vector also map final vectors back to scores over the vocabulary (Press & Wolf, 2017; Vaswani et al., 2017). A vocabulary of 100,000 tokens produces 100,000 logits at that position. Softmax then turns z into next-token probabilities.