How Does ChatGPT Actually Read?
Understanding LLMs, Part 1 of 3: Tokens & Vectorization
My old roommate Nate had a problem.
He wanted to become a certified sommelier, one of the most grueling professional exams in the hospitality world, and he had about eight months to pull it off. Every evening, without fail, he’d sink into our beat-up couch in our San Francisco apartment, mute whatever we were watching on TV, and work through a stack of flashcards so tall it looked structurally unsound.
I watched him do this for months at our apartment on the corner of Octavia and Greenwich, which of course we (I mean Nate) naturally nicknamed “the OG”. At the time, I thought it was obsessive (the flashcards, not the nicknaming). In retrospect, he was accidentally doing something remarkable. He was training his brain the same way the most powerful AI systems in the world get trained.
He just didn’t know it yet, and neither did I.
Before You Can Learn Anything, You Have to Name It
Nate’s first challenge wasn’t memorizing wine. It was breaking wine down into pieces small enough to study.
Every grape variety got a card. Every appellation. Every vintage rule, every flavor profile, every food pairing convention. Hundreds of discrete, isolated units of information representing a single entry point in the enormous web of knowledge a sommelier needs to hold in their brain.
This is extremely similar to what a large language model does before it processes a single word you type.
It’s called tokenization, and it’s the first thing that happens every time you send a message to ChatGPT.
When you type “I’d love a glass of wine,” the model doesn’t see a sentence. It sees a sequence of tokens — small chunks that might be whole words, partial words, or punctuation marks. The exact breakdown depends on the model, but that sentence might look something like this:
“I” / “’” / “d” / “ love” / “ a” / “ glass” / “ of” / “ wine”
Yes, it’s a little ugly. The model isn’t trying to preserve your grammar, it’s trying to isolate every meaningful unit it can work with. Like Nate’s flashcards, each token is a discrete unit the system can work with. You can’t learn what you can’t isolate.
Cards Aren’t Enough — You Need a Map
About three months in, Nate started doing something different with his flashcards. He stopped drilling them in isolation and started grouping them — not alphabetically, not by region, but by relationship.
A Burgundy and an Oregon Pinot Noir aren’t the same wine, but they’re made from the same grape, share a similar flavor profile, and if you understand one deeply, you already have a head start on the other. A Riesling and a Cabernet Sauvignon, on the other hand, are practically different planets. They are materially different grapes from different climates, and they produce dramatically different wines as a result.
Nate wasn’t just memorizing facts. He was building a mental map where proximity meant similarity. Wines that shared characteristics lived close together in his mind. Wines that didn’t, lived far apart.
Large language models do something structurally identical. It’s called vectorization, and it’s the step that transforms tokens (chunks of words or phrases) from isolated units into something the model can actually reason with.
The core idea is that every token gets converted into a list of numbers. This is not a random list, but rather a carefully calculated list that encodes where that word sits in relation to every other word the model has ever encountered.
In mathematical terms, that list of numbers is called a vector, and if you’ve ever taken a linear algebra class, this will sound familiar. The space where all these vectors live is called an embedding space. You can think of an embedding space as a highly complex coordinate system that serves as a map of relationships between each and every token.
Think of it like a map of the entire English language, where:
“King” and “Queen” are close together
“Wine” and “grape” are neighbors
“Wine” and “spreadsheet” are very, very far apart
The distance between any two words on that map reflects how related they are in meaning, context, and usage. A sommelier who hears “Burgundy” immediately thinks “Pinot Noir”, but this isn’t because they memorized a rule, but because the two concepts are adjacent in their mental map. A Large Language Model (LLM) does the same thing with vectors.
What makes this genuinely clever is that the coordinates capture nuance, not just category. “Bank” as in a financial institution and “bank” as in a riverbank are the same word, but in an embedding space, they end up in different neighborhoods because the contexts in which they appear are different. The map isn’t just about what words are. It’s about how they behave.
What This Means When You Hit Send
The next time you open ChatGPT and type a question, here’s what’s happening in the first fraction of a second:
Your words are broken into tokens (discrete, manageable units). Each token is converted into a vector, which is a set of coordinates that encodes its meaning and its relationship to every other word in the model’s experience. And that rich, multi-dimensional map becomes the raw material for everything the model does next.
Nate didn’t pass his sommelier exam by memorizing 500 isolated flashcards. He passed it by building a map — a mental model where everything connected to everything else, and new information had a place to land.
The model learned language the same way, card-by-card and coordinate-by-coordinate, until the map was dwas iled enough to navigate anything you typed.
You now understand how an LLM reads, and this is where we will wrap up on Part 1.
Part 2 is about something harder: how it learns.
Building the map is one thing, but training a system to build it accurately, across billions of words, in hundreds of languages, over months of computation, is something else entirely. And it starts with the same mechanism your brain uses every time you get something wrong.
Check our Part 2 now » read here.