Transformers Demystified

Artificial intelligence (AI) tools and large language models (LLMs) are everywhere. Ever since my first conversation with ChatGPT, I've been fascinated by what happens behind the scenes: how does an LLM turn my input into a coherent response? Is there a human secretly typing away in the background? Can ChatGPT magically read my mind?

These questions sent me down the path of exploring how LLMs work under the hood. But most explainers I found had the same problem: they were either too shallow to be satisfying or too technical to be approachable. This piece is my attempt to bridge that gap. The goal here is not to dive into machine learning theory or advanced AI concepts. Instead, I want to outline what actually happens to the text after you send it to an LLM, and why each step is necessary.

Transformer vs LLM

Machine learning has been around since the 1950s, and neural networks are one important type of machine learning model. In 2017, a new neural network architecture called the transformer — the “T” in ChatGPT — became the foundation for the modern LLMs we use today.

At their core, LLMs are prediction machines. More specifically, they predict the next token — a token might be a word, part of a word, a punctuation mark, or even a space. Given a block of text, the LLM converts this text to tokens and estimates which tokens are most likely to come next. Then it repeats that process again and again, generating sentences, paragraphs, and eventually full responses. So how is a transformer different from an LLM?

First, think of a factory. Now imagine ChatGPT is the entire factory. Inside that factory are one or more trained machines or LLMs, such as GPT-5.5 or GPT-5.4. Each LLM is like a specialized production machine for language. At the heart of that LLM machine is the transformer: the neural network architecture that processes numbers and produces scores for what token might come next.

ChatGPT is hence the full factory—to function as a complete app, it must wrap the transformer-powered LLM machine with an assembly line of other vital components, including system prompts, safety layers, memory features, context handling, tool integrations, and the user interface.

Basic Flow

Step 1: LLM input

You type your prompt – 'Sun rises in the' – into the LLM chat box and hit send.

Step 2: Text to Tokens

First, the tokenizer springs into action. The tokenizer is a text preprocessing component used with the LLM. It breaks down the sentence into smaller pieces called tokens. A token is a small piece of text — usually a word, part of a word, a punctuation mark, or a space-like unit. Roughly, one token is said to be about three-quarters of a word.

Step 3: Tokens to Token IDs

Each token (e.g., "Sun") is then converted to a token ID (e.g., 3457), which is a number assigned to that token by the tokenizer. The transformer does not deal with words, rather it deals with numbers. Every token in the tokenizer’s dictionary has its own unique token ID. Size of the LLM tokenizer dictionary is decided by the LLM builders.

Step 4: Representing meaning with numbers

A token ID is just a label. By itself, it does not carry any meaning. The number 3457 does not tell the model anything about the idea of "Sun"; it simply tells the model where to look.

The LLM uses that token ID to look up a row in a large table of numbers called the embedding matrix. You can think of this matrix as a giant lookup table. Every token in the tokenizer’s vocabulary has its own row, and each row contains a list of numbers. That list of numbers is called a token embedding or token vector. During training, the LLM learns the numbers in this embedding matrix. Over time, tokens that appear in similar contexts tend to get similar vectors. For example, the vector for "Sun" may end up closer to the vector for "moon" than to the vector for "banana".

So when the model sees token ID 3457, it does not treat 3457 as meaningful on its own. It uses 3457 like an address, retrieves the matching token vector, and sends that vector into the transformer for further processing.

One important detail: the token vector is only the starting numerical representation of the token. As it passes through the transformer layers, its values are updated based on the surrounding words. If the same token appears twice in the same input sequence, both occurrences usually start with the same token embedding. But after the transformer layers process the sequence, each occurrence can end up with a different contextual meaning because the surrounding words are different.

For example: I sat by the bank. I saw a duck on the water. Then, I deposited money in the bank. I then opened a new account.

In this sequence, the word "bank" appears twice. Each "bank" occurrence would usually start with the same initial token vector, because it is the same token. But after the transformer processes the full sequence, each "bank" token vector will end up with a different contextual meaning. So the first “bank” is likely a riverbank because of “duck” and “water” while the second "bank" is likely financial bank, because of "deposited money" and "account". This is one of the key jobs of the transformer: it turns a token’s basic starting meaning into a richer, context-aware meaning.

Step 5: Adding positional info

Word order matters. "Man eats egg" means something very different from "Egg eats man". To fix this, the LLM adds a second set of numbers to the token vectors that denote each token's position in the sentence. Now every token carries two things: what it means, and where it sits. Different LLMs do this in different ways, but the goal is the same: to give the LLM a sense of order.

Step 6: Transformer Blocks

This is where the main computation happens. The original token vectors now pass through a stack of transformer blocks. In this example architecture (see figure on the left), there are 12 transformer blocks, that process token vectors one after another. The number of transformer blocks is decided by the model builders.

All the transformer blocks have the same general structure, but each block has its own parameters or weights – these are internal numerical values learned during LLM training that control how each transformer block transforms the token vectors. As the token vectors move through the transformer layers, their values get updated becoming more contextual and more useful for predicting what token should come next.

Here’s some background info that happens during LLM training and not in the current flow while the LLM is being used. During training, the LLM repeatedly sees examples like:

The sun rises in the ___

LLM predicts a next token. Maybe at first it predicts something bad:

The sun rises in the banana.

The training process compares the LLM’s prediction with the actual next token in the training data: east

Then the LLM adjusts its internal weights slightly so that next time, the LLM is more likely to predict a better continuation. Over billions or trillions of examples, the LLM discovers that to predict text well, it must internally track certain patterns. Some of those patterns correspond to things we call grammar, tone, style, facts, etc. These patterns become encoded in the LLM’s learned weights and in the changing token vectors as they pass through the layers.

Now let’s continue our journey and look inside one transformer block in Steps 7 to 12. In case this section feels slightly heavy— the big idea is that context gets added and refined, layer by layer.

Step 7: Transformer – stabilize numbers

In many transformer designs, before further calculation, the LLM normalizes the token vectors. Some numbers in the token vectors can be too high or too low. Normalization is a process of adjusting these numbers to a standardized scale so that the transformer can focus on extracting the meaning rather than reacting to how big or small the numbers are.

Step 8: Transformer – attention mechanism

And now presenting - the star of the show: attention.

Attention is a mechanism which lets each input word (remember words are represented by token vectors at this point) "look at" other input words and determine which other words matter more to itself. Once the importance is determined, each word updates its values in the token vector. In our example, “Sun rises in the,” the final token “the” might determine that tokens like “Sun,” and “rises” are important in predicting the next token. So the token vector values for “the” will be updated in a way that is useful for predicting the next token.

Masked means each token is only allowed to consider itself and the tokens that came before it. This matters because the LLM’s goal is to predict what comes next, so it should not be allowed to peek ahead. In our example,

"Sun" can only see "Sun" and not the words that come after, i.e., “rises”, “in”, “the”
"rises" can only see "Sun rises"
"in" can only see "Sun rises in"
"the" can only see "Sun rises in the"

So the final token “the” has the fullest view of the sentence so far, which is why its updated token vector is especially important for predicting the next token.

Multi-head simply means the LLM does this attention calculation several times in parallel, with each "head" focusing on different kinds of relationships such as nearby words, grammatical structure, repeated concepts, or longer-range dependencies. Calculations from parallel processing are then combined for each token vector.

Step 9: Transformer – retaining original values

The LLM takes the result of the attention step and adds it back to the token vectors from before the attention step. This helps stop the original information from getting lost as the token vectors pass through many transformer layers. Instead of replacing the old token vector completely, the LLM adds the new attention-based information on top of what was already there.

Step 10: Transformer – stabilize numbers

Another tidy-up of the numbers, exactly like Step 7, to keep numbers stable before the next stage.

Step 11: Transformer – feed-forward neural network

Each token’s updated vector now passes through a neural network (called feed-forward neural network) inside the transformer block. During attention, the token vectors were updated by mixing in information from other relevant token vectors. It does not compare token vectors with each other again. That already happened during attention. Instead, it takes each token’s current vector and transforms it further, strengthening useful signals, weakening less useful ones, and preparing the token vector for the next part of the transformer.

Step 12: Transformer – retaining original values

New token vector values are added back to the values of token vectors at the end of Step 9, so useful information is preserved as we move along. And that's one full transformer block. Remember - this whole sequence (Steps 7 to 12) repeats 12 times, with the output of one transformer block flowing straight into the next.

Step 13: Stabilize numbers

Same action as Steps 7 and 10. After the final transformer block, the numbers get one last tidy-up to make sure they're in good shape for the closing steps.

Step 14: Token vectors to scores

Now based on the final vector token values in the previous step, the LLM calculates and updates a score for every token in its dictionary. In our example, "east" gets the highest score (4.3), comfortably ahead of "today", "west", and the rest.

Step 15: Pick the next token

The LLM picks the winner. Often that's simply the highest-scoring token, but LLMs can also "sample" every so often choosing a slightly lower-scoring token to keep their answers varied and natural rather than robotic. Here, it picks "east".

Step 16: LLM response

Finally, the chosen token is converted back into readable text. The LLM then appends the chosen token to the existing text so the user input "Sun rises in the" becomes "Sun rises in the east". The LLM can also simply output “east”.

Subsequent Loops

One important thing to know: the LLM generates only one token at a time.

To write a full sentence or paragraph, it adds the newly chosen token to the text so far, then repeats the prediction process to choose the next token, and then the next token, and then the next.

In practice, modern LLMs use shortcuts so they do not recalculate all the previous token information from scratch each time. But the basic idea remains the same: the response is built one token at a time. The loop continues until the LLM reaches a stopping point, predicts an end token, hits a limit, or is stopped by the system.

So the next time ChatGPT, Claude, or Gemini replies to you, you’ll have a better sense of what is happening under the hood:

• Your text is broken into small pieces called tokens
• Each token is turned into a number the LLM can work with
• These numbers pass through transformers and are updated based on context
• Using that context, the LLM scores possible next tokens
• LLM picks one token, adds it to the response, and repeats the process
• LLM converts these tokens back into readable text giving you the reply

Not magic — just math, repeated one token at a time.