#tokenization

tokenization shows up across 1 section(s) and 1 page(s) in this workspace. Use this page as a topic map, not just an archive.

Start here

If you are new to this topic, begin with the strongest entry points first, then move into related notes and supporting material.

Where it appears

  • Systems 1 page(s)
systems

From Text to Tokens

Explaining tokenization without the math. Why models don't read words, and why that matters for your prompts.