Essays
Tagged "tokens"
2 essays touching on tokens.
Artificial Intelligence
Tokens Aren't Words
An LLM doesn't read your sentence — it reads a string of token IDs handed to it by a tokenizer that doesn't care about spelling or place value.
An LLM doesn't read your sentence — it reads a string of token IDs handed to it by a tokenizer that doesn't care about spelling or place value. That's the real reason it can't count the r's in strawberry, and why big-number math goes sideways.
Surya · 4 min read
Artificial Intelligence
PUSHED OUTIN CONTEXT WINDOW
Claude Sonnet 5
~1M
GPT-5.1
~400K
Gemini 3 Pro
up to 1M
100 TOKENS ≈ 75 WORDSNOT MEMORY — A BUDGET
Context Windows Aren't Memory
A larger window is not the same as remembering.
A context window isn't a brain that remembers your conversation. It's a token budget — and once it's full, the oldest content doesn't fade, it just stops existing to the model.
Surya · 3 min read