LLM explained by LLM

This is a little visual guide on how LLM models are built and trained, also many parts are just general deep learning concepts.

by v1e48a74dejs1 min

llmdeep learningneural networksactivation functionsloss functiongradient descentbackpropagationtokenizationembeddingstransformer

open full screen ↗

article

Part A

The brain-ish machine

01

Neurons, weights and biases

A neuron multiplies each input by a weight (how much it cares), adds the results, adds a bias (its default mood), and passes the total on.

Think of it as a judge scoring a dish: taste counts a lot, price counts against it, plus a personal generosity bonus.

Show me the math

The weighted sum is a dot product: multiply matching numbers, then add them up.

z = w₁x₁ + w₂x₂ + w₃x₃ + b Common mix-up
  • “Artificial neurons work like brain cells.” They are only loosely inspired by biology. Each one is a small arithmetic step, and the knowledge sits in the weights, not in any single neuron.
02

Layers and networks

Put many neurons side by side to make a layer, then chain layers, so each one turns the previous layer's numbers into slightly more useful numbers.

Think of it as an assembly line, or a team of detectives: the first desk spots edges, the next spots shapes, the last decides “that's a 7”.

Scale check 3Blue1Brown's digit reader takes 784 pixels in and gives 10 numbers out. With two hidden layers of 16 neurons it has 13,002 weights and biases.

Show me the math

A whole layer is a matrix times a vector: many dot products computed at once.

Common mix-up
  • “Engineers program what each layer does.” They only choose the shape. What each layer ends up doing is learned from data during training.
03

Activation functions

After adding up its inputs, each neuron bends the total with a simple rule, such as ReLU: “if negative, output 0”. Bends let the network learn curves instead of only straight lines.

Think of it as a dimmer switch with a floor. Below zero the light stays off; above zero it gets brighter in step.

Show me the math ReLU(x) = max(0, x)

Sigmoid squashes any number into 0 to 1. Tanh squashes it into −1 to 1. Google's course notes that ReLU often works a little better than both, partly because it is less prone to the vanishing gradient problem and is cheap to compute.

Common mix-up
  • “Activation functions are optional decoration.” Without them, 100 stacked layers collapse into one straight-line function, so depth would add nothing.
Part B

How it learns

04

Loss function

The loss is one number that says how wrong a guess was. Big loss means a bad guess. Zero means perfect.

Think of it as a golf score: lower is better, and the whole game is bringing it down.

Show me the math

For LLMs the loss is cross-entropy: the negative natural log of the probability the model gave to the right token. Small probability on the right answer means a big loss.

loss = −ln(pright) Common mix-up
  • “Loss is accuracy.” Loss measures how much probability the model put on the right answer. Picking the right word while being only 30% sure of it still costs about 1.2.
05

Gradient descent and backpropagation

Training asks, over and over: which small change to each weight would lower the loss? Backpropagation works that out. Gradient descent then nudges every weight a little that way. Repeat billions of times.

Think of it as walking downhill blindfolded in fog. You feel the slope under your feet and step downhill. The step size is the learning rate. Backpropagation is the efficient way to feel the slope for millions of dials at once, working backward from the mistake to see who is to blame.

Show me the math

The derivative is a slope. The gradient is all the slopes at once, and it points uphill, so we step the opposite way. The chain rule lets backpropagation reuse each result as it moves backward.

w ← w − learning_rate × slope Common mix-ups
  • “Backpropagation and gradient descent are the same thing.” Backpropagation works out the direction. Gradient descent takes the step.
  • “The model learns while I chat with it.” Its weights are frozen after training. A conversation does not change them.
Part C

Turning words into numbers

06

Tokens and tokenization

Before the model sees text, it is chopped into tokens (common words, word pieces, or single bytes), and each token gets an ID number.

Think of it as LEGO bricks. Common words come as one big pre-built brick (“the”). Rare words are snapped together from smaller bricks (“st” + “raw” + “berry”).

Rule of thumb For English, one token is about 4 characters or 0.75 words (OpenAI docs). GPT-2's vocabulary holds 50,257 tokens.

Go deeper

Byte-Pair Encoding (BPE) starts from single characters and repeatedly merges the most frequent neighboring pair. The stepper above runs that loop on a tiny corpus.

Common mix-up
  • “The model sees letters and words like we do.” It sees token IDs. That is a standard explanation for why models stumble on counting the r's in “strawberry”, which GPT-4o's tokenizer splits as st · raw · berry. One study (Fu et al., 2024) found models can recognize the letters but not count them, so tokenization is part of the story, not all of it.
07

Embeddings

Each token ID becomes a long list of numbers, a point in a “meaning space” where similar words sit close together.

Think of it as a map where words are addresses. “Cat” lives next door to “dog” and far from “bus”, and directions on the map carry meaning.

Scale check This map has 2 numbers per word. GPT-3 uses 12,288.

Show me the math

A vector is a list of numbers, read as a point or an arrow. “Close” is measured by distance, or by the dot product.

Common mix-ups
  • “king − man + woman = queen, exactly.” The result only lands near “queen”, and queen is the closest word in the vocabulary.
  • “Each number in a vector means one human concept.” Mostly they do not. Meaning is spread across many numbers.
  • Inside a transformer, a word's starting embedding is later reshaped by context, such as the “bank” of a river versus a money “bank”.
08

Next-token prediction

An LLM's whole job: look at the text so far, give every possible next token a probability, pick one, add it to the text, and repeat.

Think of it as phone autocomplete with a far bigger brain, or as finishing a torn movie script one word at a time.

Show me the math

Softmax turns any list of raw scores into probabilities that are positive and add up to 100%. The model itself is deterministic. A prompt gives different answers across runs because the pick is random, weighted by those probabilities.

Common mix-ups
  • “It looks answers up in a database or on the internet.” A plain model predicts from patterns stored in its weights.
  • “Hallucinations are random glitches.” Pretraining has no true/false labels, but OpenAI's 2025 analysis argues the larger cause is that standard training and evaluation reward guessing over admitting uncertainty. So “next-token prediction causes hallucination” is too simple.
Part D

The transformer

09

The transformer and attention

A transformer lets every token look at the tokens before it, decide how much each one matters (attention), and blend in their information. Many layers repeat this, and all tokens are processed in parallel.

Think of it as a party. Each word holds up a query (“who describes me?”), wears a key name-tag (“I'm an adjective”), and offers a value (what it can tell you). When a query and a key match well, you listen more to that word's value. Softmax turns the match scores into percentages.

Scale check GPT-3 runs 96 attention heads per layer across 96 layers. Its MLP blocks, not attention, hold about 116 billion of its 175 billion parameters (3Blue1Brown).

Show me the math Attention(Q, K, V) = softmax(QKᵀ / √d) · V

Q·Kᵀ is a grid of dot products (query against every key). Dividing by √d keeps the scores from growing too large. Softmax turns each row into percentages. Multiplying by V blends the values using those percentages. Q, K and V come from three learned matrices.

Common mix-ups
  • “Attention means the model understands or focuses like a person.” It is a learned weighted average.
  • “Q, K and V are hand-written rules like ‘find the adjective’.” They are learned. The adjective story is a teaching device, and the vectors in the demo above are hand-picked for illustration.
  • “A transformer is all attention.” The MLP (feed-forward) blocks hold most of the parameters.
Part E

Making a chatbot

10

Pretraining on large text

The model starts with random weights and spends months reading a huge amount of text, playing “guess the next token” and being corrected by backpropagation each time.

Think of it as reading every textbook in the library. In Andrej Karpathy's textbook analogy, pretraining is the reading, fine-tuning is the worked examples, and reinforcement learning is the practice problems.

Scale check A person reading non-stop would need over 2,600 years to get through GPT-3's training text (3Blue1Brown). The result is a base model, an internet-text simulator that continues documents but is not yet a helpful assistant.

Go deeper

For the full stack in one sitting, Andrej Karpathy's video “Deep Dive into LLMs like ChatGPT” (about 3.5 hours) covers pretraining data, tokenization, the network, inference, fine-tuning and reinforcement learning.

Common mix-up
  • “The model stores a copy of the internet.” It compresses statistical patterns into its weights. It can recall some text verbatim, but it is not a searchable archive, which is also why its knowledge can be fuzzy or out of date.
11

Fine-tuning (instruction tuning)

The base model is trained further on a smaller, carefully written set of example conversations (question, then ideal answer), so it learns to behave like a helpful assistant.

Think of it as an apprentice who has read the whole library and now studies worked examples from a master: “when asked this, answer like that.”

Go deeper Common mix-up
  • “Fine-tuning teaches the model most of what it knows.” It mostly teaches format and behavior. The knowledge comes from pretraining.
12

RLHF: learning from human feedback

People compare pairs of answers and pick the better one. A reward model learns to predict those choices, and the LLM is then tuned to write answers the reward model scores highly.

Think of it as a cooking show. Judges taste two dishes and point to the better one, a sous-chef learns to predict the judges' taste, and the cook practices against the sous-chef thousands of times.

Why it matters In OpenAI's InstructGPT study, people preferred answers from a 1.3-billion-parameter model tuned with human feedback over answers from the 175-billion-parameter GPT-3, despite the 100× size gap.

Go deeper Common mix-ups
  • “Humans rate every answer the model gives.” They rate a sample, and the reward model generalizes from it.
  • “RLHF makes the model truthful.” It aligns the model with what raters prefer. The InstructGPT paper itself notes the model still makes simple mistakes.
  • “RLHF is the only post-training method.” Newer pipelines also use reinforcement learning on checkable tasks such as math and code, and alternatives such as DPO.
Part F

Talking to it

13

Inference and sampling

When you chat, the frozen model produces probabilities for the next token. A sampling rule picks one, and the loop repeats until a stop token. Temperature controls how bold the pick is.

Think of it as a spice dial. Low always orders the house favorite. High tries unusual dishes. Top-p says “only consider the dishes that together make up the top slice of popularity”.

Show me the math P = softmax(logits ÷ T)

Below 1, dividing sharpens the distribution. Above 1, it flattens it. Top-p (nucleus sampling) keeps the smallest set of tokens whose probabilities add up to p, then renormalizes.

Common mix-ups
  • “Higher temperature makes the model smarter or more creative in its thinking.” It only changes the dice roll over the same probabilities.
  • “Temperature 0 makes it correct.” It makes it consistent. The top-probability answer can still be wrong.
  • “The model plans the whole answer, then types it.” It generates one token at a time, and each new token is fed back in as input.

The whole machine on one screen

Math notions, in plain words

NotionPlain wordsPrimary linkBackup link
Dot productMultiply matching numbers and add: how much two arrows point the same way.3Blue1BrownKhan Academy
VectorA list of numbers, read as an arrow or a point.3Blue1BrownJay Alammar
MatrixA grid of numbers that turns one list of numbers into another.3Blue1Brown3Blue1Brown
DerivativeThe slope: how much the output changes when you nudge the input.Khan AcademySimple Wikipedia
GradientAll the slopes at once. It points uphill, so we step the other way.Khan AcademyWikipedia
Chain rulePassing blame backward through each step of a calculation.3Blue1Brown3Blue1Brown
Activation (ReLU, sigmoid)A bend that lets networks learn curves.Google ML Crash CourseWikipedia
SoftmaxTurns any scores into percentages that sum to 100%.Wikipedia3Blue1Brown
Loss (cost)One number for “how wrong”.3Blue1Brown3Blue1Brown

Go deeper, live

TensorFlow PlaygroundTrain a real small network in your browser, no code. Transformer ExplainerA live GPT-2 small model with a temperature slider. LLM VisualizationA 3D tour of a tiny GPT, next to GPT-2 and GPT-3. 3Blue1Brown deep learning seriesThe best single spine for everything on this page.

What this page simplifies

  • Every number in the widgets is a toy value unless a source is named. Real embeddings, vocabularies and networks are vastly larger.
  • Token counts depend on the tokenizer. The 4-characters-per-token rule is for English with OpenAI tokenizers. Code and other languages use more.
  • Pretrain, then fine-tune, then RLHF is the classic recipe. Many labs now add reinforcement learning on checkable tasks and preference methods such as DPO.
  • GPT-3 figures are illustrative. Current frontier models do not publish such details.

open this node in the explorer →