The idea
- Training: the model reads vast amounts of text and adjusts billions of internal parameters to predict better. It's expensive, slow, and done by the AI lab.
- Inference: running the trained model on a request. Every API call your product makes is inference. The model is frozen. Nothing users type changes it.
Training stops at a date, so the model's built-in knowledge has a knowledge cutoff.
Mental model
A brilliant contractor who read everything up to a certain date and has amnesia between meetings. Anything newer, or specific to you, you hand them in the briefing.
Why it matters when you build
- Users correcting the AI doesn't teach the model. It helps only within that conversation. For a product that improves, save the corrections and feed them back into context as examples or memory.
- Fresh or private knowledge comes through context. That includes your docs, today's data and current prices. This is what retrieval (RAG) is for, covered in module 2.
- Fine-tuning exists (extra training on your data), but it's rarely the first tool to reach for. Better context usually wins and is far cheaper to change.
Gotchas
- Models can sound confident about events after their cutoff. If recency matters, provide current information and tell the model to rely on it.