Training, Fine-Tuning, and Inference
Inference vs fine-tuning vs training: what changes the model itself, and what only changes what it sees in one run. That decides training or better context.
Someone asks for a model trained on their documents. Usually what they want is for the model to be able to use those documents, which is a different thing and a much cheaper one. Telling the two apart needs one distinction: what changes the model, and what only changes what the model sees.
- What changes the model
Writes the parameters
- Pre-training, over a very large text corpus, weeks to months
- Fine-tuning, over a smaller curated set, hours to days
Reads them only
- Inference, over your input, milliseconds to seconds
- Everything supplied at request time
- What changes the model
Two stages write to the parameters. The third only reads them.
Training
Pre-training produces the parameters. A model is shown enormous quantities of text and repeatedly adjusts its internal numbers to get better at predicting what comes next. The output is a set of weights.
This is the expensive stage, and almost nobody outside a handful of labs does it. It takes specialised hardware running for a long time.
Fine-tuning
Fine-tuning continues that same process on a smaller, curated set of examples, with a narrower objective. It writes to the parameters, exactly as pre-training does. The difference is scale and intent, not mechanism.
You will see it said that fine-tuning changes behaviour rather than knowledge. Treat that as a rule of thumb rather than a law. Fine-tuning is further training, and further training on a body of material can change what a model produces about that material. The honest version is narrower: fine-tuning is usually a poor and expensive way to get specific, changing facts into a system, and it is often a good way to fix a consistent format, register or task pattern that is awkward to specify in an instruction every time.
There is also a real cost. Training further on a narrow set can make a model worse at things it used to do well.
Usually a poor fit
- Specific, changing facts
- Expensive, and can make the model worse at what it did well
Often a good fit
- A consistent format or register
- A task pattern awkward to specify every time
It writes to the parameters, exactly as pre-training does.
Inference
Inference is what happens when you send input and get output. The model reads its parameters and does not write to them. The same weights that answered the last request answer this one.
Context is not training
Everything you supply at request time, the instruction, the conversation so far, any documents the application pasted in, is input. It shapes the output, sometimes dramatically. It does not change a single parameter.
This is the distinction people most often lose:
| Changes parameters | Changes what one run sees | |
|---|---|---|
| Pre-training | yes | no |
| Fine-tuning | yes | no |
| Prompting | no | yes |
| Retrieval, pasted documents, conversation history | no | yes |
Four different mechanisms get flattened into the single word "learning", and that flattening is where bad architecture decisions start.
Does it remember?
Not in the file. Nothing you type adjusts the weights of a hosted model as you talk to it.
But applications built around a model can absolutely remember, by storing what you said and putting it back into the input next time. When an assistant recalls your name a week later, some code wrote it down and some code retrieved it. That memory lives in the application, and you can inspect it, edit it and delete it, precisely because it is not in the model.
One caveat so the rule stays accurate rather than absolute. Research systems and some specialised setups do update parameters during operation, and continual-learning methods exist. The point is not that run-time parameter updates are impossible. It is that mainstream hosted models do not do it, so if you are using one, assume the weights are fixed until you have specific evidence otherwise.
- Where it remembers
The application
- Stores what you said
- Puts it back into the input next time
- You can inspect it, edit it, delete it
The parameters
- Not changed by what you type, for mainstream hosted models
- Where it remembers
That memory lives in the application, and you can inspect, edit and delete it, precisely because it is not in the model.
The decision this usually settles
When you want a model to work with information it did not have, ask what kind of information it is.
If the facts are specific, private, or liable to change, put them in the input. Retrieve them and supply them at request time. You can then update them by editing a document rather than by running a training job, and you can show the reader where the answer came from.
If the problem is that the output keeps coming back in the wrong shape or register, and no reasonable instruction fixes it, fine-tuning is worth considering.
Most of the time the request for training is really a request for better context, which is cheaper, faster to change, and easier to inspect when it goes wrong.
What kind of information is it?
- Specific, private, or liable to change
Put it in the input: retrieve and supply it at request time
Update a document instead of running a training job.
- The output keeps coming back in the wrong shape or register
Fine-tuning is worth considering
For what the file itself is doing, see what an AI model actually is. For the retrieval side of supplying context, see retrieval-augmented generation in plain terms.
Where this helps, and where it stops
- What it is
- A conceptual explainer separating the stages that write model parameters from the stages that only supply input.
- What it does not guarantee
- A guide to running a training or fine-tuning job, and not a cost comparison between specific providers.
- When the distinction matters
- Someone proposes training a model on internal documents.