Tokens

2 stories tagged Tokens on Learning Fanatic.

Lesson 1.2: Tokens, context and why wording matters Learn what tokens and context windows are, how to budget the text you send, and how word choice and order quietly change what a model gives back.
Lesson 2.1: Tokens and context windows explained Language models read tokens, not words, and can only hold so many at once. Learn what tokens are, how to estimate them, and why the context window matters.