The idea
A large language model (LLM) does one thing: given some text, it predicts what comes next. It generates a response by predicting one token, appending it, and predicting again, over and over until it stops.
It is autocomplete. What makes it remarkable is scale: to predict the next line of a good spec or working code, the model has to learn a great deal about specs, code and the world. The reasoning you see is a side effect of being very good at prediction.
Mental model
Someone sketching in pen. Each stroke is committed; they can't go back and erase, only keep drawing.
Why it matters when you build
- Streaming is the real mechanism. Text appears word by word because it is generated in that order. Show it as it arrives instead of waiting for the whole answer.
- No going back. The model can't revise what it already wrote mid-answer. If you want a plan-then-answer or draft-then-revise flow, design it as separate steps.
- Order matters. Asking for reasoning before the final answer often produces better answers than asking for the answer first.
Gotchas
- "It's just autocomplete" is true and misleading at the same time. Don't underestimate it, and don't treat it as a person either.