What a language model is, and why it sometimes makes things up
ChatGPT, Gemini and Claude are all built on language models. Here is how they really work, with no formulas, and why they occasionally get things wrong with total confidence.
- Level
- beginner
- Time
- 5 minutes
- What you need
- Nothing, just curiosity
When you type a question into an assistant like ChatGPT, Gemini or Claude, there is no search engine on the other side, and no encyclopedia either. There is a large language model, or LLM: a program that has learned to predict which word comes after another by reading an enormous amount of text.
A game of completion
The principle is the same as your phone keyboard’s autocomplete, scaled up massively. The model receives a sequence of words (or rather word fragments, called tokens) and calculates the probability of each possible next token. It picks the most fitting one, appends it to the sequence and starts again. A ten-line answer is the result of hundreds of these predictions in a chain.
What makes modern models so capable is the Transformer architecture, introduced in 2017 by a group of Google researchers. Its key mechanism, attention, lets the model weigh which words in the preceding text matter most when deciding the next one. That is how it keeps track of a long argument, respects grammar and picks up context.
Where it learns
Training happens on billions of pages: books, websites, articles, code. The model does not memorise the text word for word; it adjusts billions of internal parameters until its predictions become accurate. After that phase, developers refine it with example conversations and with feedback from human reviewers, to make it useful and to teach it to refuse dangerous requests.
Why it “hallucinates”
A language model has no archive of facts to consult: it only has probabilities. Ask it the birth date of a little-known person and it will still produce a plausible date, because “a date” is what statistically follows that kind of question. The result is an answer written with the same confidence as the correct ones, but invented. This is called a hallucination, and it is the most important limitation to keep in mind.
To reduce the problem, many assistants now search the web or documents you provide before answering, and cite their sources. A simple rule still applies: for numbers, dates, names and quotations, always check an independent source.
The takeaway
An LLM is an extraordinary tool for writing, summarising, translating, coding and reasoning about text you give it. It is less reliable as an oracle for specific facts. Using it well means knowing which of the two jobs you are asking it to do.