# Next-token prediction

Concept · [Building with AI](https://ecosystem.samcarlton.com/ai)

Also called: predicting the next word, autoregressive, autoregression, text prediction, next word prediction, autocomplete

The one job a [language model](https://ecosystem.samcarlton.com/llm) performs. Given the text so far, it picks a likely next [token](https://ecosystem.samcarlton.com/token), adds it, and repeats. Every answer, plan or piece of code comes out one token at a time this way, which helps explain why models sound fluent and why they can state wrong things with confidence.

**For example:** When you ask a [chatbot](https://ecosystem.samcarlton.com/chatbot) for a tagline, it does not plan the whole line first. It builds the tagline [token](https://ecosystem.samcarlton.com/token) by token, each choice based on everything written before it.

## Connects to (8)

- [LLM](https://ecosystem.samcarlton.com/llm): Large language model.
- [Token](https://ecosystem.samcarlton.com/token): In AI, a small chunk of text, roughly a word or a piece of a word, that an LLM reads and writes.
- [Hallucination](https://ecosystem.samcarlton.com/hallucination): When an AI model states something false or invented as though it were true.
- [Probabilistic vs deterministic](https://ecosystem.samcarlton.com/probabilistic-vs-deterministic): Deterministic software gives the same output for the same input every time.
- [Training](https://ecosystem.samcarlton.com/training): Teaching an AI model by showing it a very large number of examples before anyone uses it.
- [Output tokens](https://ecosystem.samcarlton.com/output-tokens): The tokens the model writes back.
- [Transformer](https://ecosystem.samcarlton.com/transformer): The design that most modern language models are built on, introduced by Google researchers in 2017.
- [Sampling](https://ecosystem.samcarlton.com/sampling): How a model chooses each next token from its list of likely options.

Map view: https://ecosystem.samcarlton.com/next-token-prediction