Next token prediction

2 stories tagged Next token prediction on Learning Fanatic.

Lesson 1.1: What a language model is, and what we will build A language model gives the probability of the next token. See one as a table of counts, score it with loss and perplexity, and plan the GPT we will build.
Lesson 2.2: How a model predicts the next token A language model works by predicting one token at a time. Understand probabilities, why answers vary, and why fluent text is not the same as true text.