Tokens and embeddings

How text becomes numbers: tokenizers, embedding vectors and meaning as geometry.

4 lessons planned · none written yet

  1. 1TokenizationNot written yetByte-pair encoding and why “strawberry” is hard.
  2. 2EmbeddingsNot written yetWords as points in space.
  3. 3Geometry of meaningNot written yetking − man + woman, and its limits.
  4. 4Positional informationNot written yetHow a model knows word order.