Pruning My Pothos
← StoryboardsStoryboard · 7 frames · Stage 01, How models work

What an AI Model Actually Is

A file of numbers that ranks what comes next, and a lot of software built around it. Seven frames on which behaviour comes from which.

Download PDFRead the article →

01 / 07

  1. A glass case labelled Exhibit A holds a single file of numbers, one eye open, while two visitors kneel in worship. A file of numbers, worshipped accordingly. It turns input into a ranking over what comes next. Retrieval, tools, memory, checks and policy are separate software that people build around it.

  2. A trained model is a file of numbers called parameters, billions of them, fixed after training. There is no row that says France, Paris: that lookup-table picture is crossed out. No single number is a fact; the numbers encode patterns that were useful for predicting text.

  3. One run of a language model: cut the text into tokens, each becoming a number. Score every token, as the parameters rank what could come next. Pick one, not always the top one, which is why wording varies. Add it and run again; that loop is what generation means. Image models, speech models and classifiers work differently.

  4. Asked the capital of France, the same steps produce Paris, which matches the world. Asked who first walked on Mars, the same steps produce a name, and no one checked. Three things get confused: how likely the model scored it, how confident it sounds, which is crossed out as a property of the wording, and whether it is true, which nothing in the loop checks.

  5. Confidence is a writing style. A confident tone is a pattern in the text the model was trained on. It says nothing about how reliable this answer is; a truth check has to come from somewhere else.

  6. The Model says: that was the app. It didn't look it up; retrieval did. It didn't refuse; policy did. It didn't remember; stored state did. What it did was rank what comes next. When the output is wrong, the model is one suspect among several, and often not the right one.

  7. Hand it the facts rather than asking it to hold them. Transforming text you gave it is a different reliability problem from recalling text you didn't. Expect variation between runs, and when something goes wrong, ask which part failed before blaming the model.