Next-token prediction
The repeated process of choosing a likely next token from all tokens already in context.
Illustrative conversation
- Does the model retrieve its whole answer from a database?
- Usually no. It generates the answer one token at a time from probabilities conditioned on the current context.
Definition
An autoregressive language model generates by estimating a probability distribution for the next token based on the tokens already in context, selecting one, and repeating the process. Complex outputs emerge from that repeated operation rather than from retrieving a complete stored sentence. Text, code, JSON, and tool-call arguments can all be produced through the same token-by-token mechanism.
Reviewed