← StoryboardsStoryboard · 6 frames · Stage 01, How models work

What an AI Model Actually Is

A file of numbers that ranks what comes next, and a lot of software built around it. Six frames on which behaviour comes from which.

01 / 06

  1. PRUNING MY POTHOSSYSTEMS · STAGE 01A STORYBOARD IN 6 FRAMESWhat an AImodelactually isEXHIBIT ATHE MODEL0.02-1.440.88...(that's the whole exhibit)a file of numbers.that's it.built around it, by people:retrievaltoolsmemorycheckspolicy
    An AI model is a file of numbers that turns input into a ranking over what comes next. Retrieval, tools, memory, checks and policy are separate software built around it.
    PRUNINGMYPOTHOS.COM01 / 06

    A museum case, roped off, labelled Exhibit A: the model. Inside is a single file of numbers, and that is the whole exhibit. It turns input into a ranking over what comes next. Retrieval, tools, memory, checks and policy are separate software that people build around it.

  2. PRUNING MY POTHOSWHAT IS IN THE FILEIt's a file.Of numbers.0.0213 -1.4470.8811 0.004-0.339 1.2170.5006 -0.0711.9031 0.318-0.772 0.064(billions more)fixed after trainingFrance →Paristhere's no such rowno single numberis a fact
    Training set the numbers. After training they are fixed, and every run uses the same ones. They encode patterns useful for predicting text, spread across billions of values.
    PRUNINGMYPOTHOS.COM02 / 06

    A trained model is a file of numbers called parameters, billions of them, fixed after training. There is no row that says France, Paris: that lookup-table picture is crossed out. No single number is a fact; the numbers encode patterns that were useful for predicting text.

  3. PRUNING MY POTHOSONE RUNOne run, one pieceat a time.whatisCut the text into tokenseach piece becomes a numberScore every tokenthe parameters rank what could come nextPick onenot always the top one, so wording variesAdd it, run againthat loop is what “generation” means
    This describes language models specifically. Image models, speech models and classifiers are models too, and they don't work this way.
    PRUNINGMYPOTHOS.COM03 / 06

    One run of a language model: cut the text into tokens, each becoming a number. Score every token, as the parameters rank what could come next. Pick one, not always the top one, which is why wording varies. Add it and run again; that loop is what generation means. Image models, speech models and classifiers work differently.

  4. PRUNING MY POTHOSPLAUSIBLE IS NOT TRUELikelyisn't true.capital of France?same stepsParismatches the worldfirst person on Mars?same stepsa name.no one checkedHow likely it scoredthe model's own rankingHow confident it soundsa property of the wordingWhether it's truenothing in the loop checks this
    A confident tone is a pattern in the training text, not a report on the model's reliability. If you want a truth check, something else has to perform it.
    PRUNINGMYPOTHOS.COM04 / 06

    Asked the capital of France, the same steps produce Paris, which matches the world. Asked who first walked on Mars, the same steps produce a name, and no one checked. Three things get confused: how likely the model scored it, how confident it sounds, which is crossed out as a property of the wording, and whether it is true, which nothing in the loop checks.

  5. PRUNING MY POTHOSWHAT IT DOES NOT DOIt didn't lookthat up.that wasthe app.separate software, each fixableretrievalstatepolicyIt looked it upretrieval did.It refusedpolicy did.It rememberedstored state did.It rankedwhat comes next.That part was it.
    When the output is wrong, the model is one suspect among several, and often not the right one.
    PRUNINGMYPOTHOS.COM05 / 06

    The Model says: that was the app. It didn't look it up; retrieval did. It didn't refuse; policy did. It didn't remember; stored state did. What it did was rank what comes next. When the output is wrong, the model is one suspect among several, and often not the right one.

  6. PRUNING MY POTHOSWHAT THIS CHANGESHand it the facts.Don't ask itto hold them.the facts,handed overreading,not recallingtransforming whatyou gave it≠ recalling whatyou didn't
    Give it the facts rather than asking it to hold them. Expect variation between runs, and when something goes wrong, ask which part failed before blaming the model.
    Read the whole explanationpruningmypothos.com/systems/what-an-ai-model-actually-is/PRUNINGMYPOTHOS.COM06 / 06

    Hand it the facts rather than asking it to hold them. Transforming text you gave it is a different reliability problem from recalling text you didn't. Expect variation between runs, and when something goes wrong, ask which part failed before blaming the model.