When you chat with an AI assistant, what happens behind the screen is not a lookup in a database of answers. Behind it is a large language model (LLM): a neural network that does not “know” anything in the human sense, but predicts the next word from a statistical pattern it has learned. Understanding how it works means looking at the Transformer architecture, the tokens and the training process.
It all starts with tokens, not words
An LLM does not read text the way you do. Its input is made of tokens: chunks of text that can be a whole word, part of one or a single character, depending on a tokenization algorithm such as Byte-Pair Encoding (BPE). BPE scans a corpus, counts the most frequent byte sequences and builds a vocabulary where common text takes one token and rare text is split into several. A long sentence can turn into dozens of tokens, and each one gets an ID number in the model’s vocabulary.
That number still means nothing to the network. Before processing it, each token is converted into an embedding: a vector of hundreds or thousands of numeric dimensions that encodes its meaning. Tokens with similar senses end up close in that vector space, so the network works with geometry rather than with strings of characters.
The Transformer architecture: attention in layers
Almost every modern LLM is built on the Transformer, the architecture Google introduced in 2017 in the paper “Attention Is All You Need”. Its core piece is the self-attention mechanism: for each token, the model computes how much it should “focus” on each of the other tokens in the sentence. That is done by generating three vectors per token —query, key and value— and comparing queries with keys using a normalized dot product. The result, after a softmax function turns it into weights that sum to one, decides what proportion of each value gets mixed into the token’s representation.
A real Transformer does not use a single attention but multiple attention heads: several parallel attentions that capture different relationships —syntax, references, distant context— whose outputs are concatenated. On top of that mix, feed-forward layers (small fully connected networks) and layer normalization are applied. Everything is stacked in dozens of blocks; each block refines the representation of every token in light of the rest. Positions are injected separately with positional encodings, because attention by itself does not know the order of words.
From context to prediction
After passing through all the blocks, each position produces a final vector. An output layer turns it into a probability distribution over the whole vocabulary: the network says “the next word is probably this one”. During generation, the model picks from that distribution —using techniques such as temperature (which flattens or sharpens the probabilities) or top-p (which restricts sampling to the most likely options)— adds the chosen token to the context, and repeats the cycle.
That is why an LLM can seem to “think”: in reality it runs an autoregressive prediction, step by step, where each new token becomes part of the input for the next one. The context limit is simply how many tokens fit in that window before attention becomes computationally unfeasible.
The cost: why you need a GPU
All that attention is expensive. Computing attention between every pair of tokens grows with the square of the sequence length, and the operations are essentially matrices and vector products. These operations run in massive parallel fashion, something GPUs (graphics processors with thousands of cores) are designed for. That is why training a large LLM requires entire clusters of GPUs for weeks, and why new-generation models constantly optimize the attention mechanism (such as sliding-window attention or flash attention) to reduce that load.
How a language model is taught
The process has three phases. First, pretraining: the model reads huge amounts of text and learns to predict the next word, adjusting its weights with backpropagation and gradient descent. In this phase it acquires grammar, facts and implicit reasoning, along with biases from the data. Next comes supervised fine-tuning: it is shown thousands of instruction-response pairs written by humans so it imitates the assistant format. Finally, RLHF (reinforcement learning from human feedback): humans score answers, a reward model learns to predict those scores and the LLM is optimized to maximize it. That stage is what aligns the model with what we expect from a helpful and safe assistant.
What an LLM is not
It is worth being honest about the limits. An LLM predicts text; it does not run exact calculations or verify facts by itself. Hence it can produce hallucinations: confident but false answers, because it generates the most probable sequence, not the correct one. Modern architectures soften this with external tools, search or explicit reasoning, but the engine remains a statistical prediction machine. Understanding that helps you use it wisely: it is a formidable tool for language and synthesis, not an infallible source of truth.
