Prompting Is Not the Skill You Think It Is

Write "always respond in JSON" in capitals, add "this is very important", and most of the time you will get JSON. Occasionally you will not. Nothing about the emphasis changed what the system does when the output is wrong, because nothing was checking.

That gap is what this page is about.

What a prompt actually is

A prompt is text placed into the model input. It joins whatever else is in there and shapes the ranking the model produces at each step.

Shaping is real. A vague request leaves a wide field of plausible continuations, and a precise one narrows it. Ask for writing about marketing and you could get anything. Ask for three slogans for a coffee brand, playful, under five words each, and the range of likely responses collapses towards what you meant.

So instructions work. The question is what "work" means.

What an instruction can express

Usefully, quite a lot:

• Intent. What you are trying to achieve, which is the part people skip most.
• Constraints. What must not happen.
• Examples. Two or three showing the pattern, which is often worth more than a paragraph describing it.
• Requested shape. The format you want back.
• Priorities. What to favour when instructions pull against each other.

Each of these narrows the field. None of them closes it.

What it cannot do

Generation is conditioned by the instruction, and no separate step afterwards verifies that the output satisfied it.

The model produces a likely continuation given everything in its input, and your requirement is part of that input rather than a rule applied afterwards. There is no step where the system notices that the response was supposed to be JSON, checks whether it is, and objects. If you want that step, you have to build it.

This is why adding force to the wording has diminishing returns. "Always" and "never" and "you must" are more input, and more input is not a check.

A prompt tells the model what you want. A contract gives software something it can check.

The difference is where a failure gets caught. An instruction alone does not create a check, so if nothing else validates the requirement, the failure can surface downstream. A validator catches it at a point you chose.

Language as the interface, contracts underneath

None of this means plain language is the wrong way in. It is often the right one. Someone can describe what they want without knowing your field names, and the system can work out the rest.

Natural language can be the human-facing interface, while explicit contracts underneath make the result dependable enough to inspect.

Worth resisting the shorter version of that, which is that language has become the API. An API has defined semantics and a contract that software can verify. Ordinary prose does not acquire those properties by being pointed at a model. The interface changed. The need for something checkable underneath did not.

The actual skill

The skill is reading the gap between what you asked for and what came back.

Not composing a better opening incantation. Looking at an output, naming precisely what is wrong with it, and choosing the right repair:

• Wrong tone, wrong length, wrong register: usually an instruction problem. Say the thing you did not say.
• Wrong shape, inconsistently: an instruction can improve the odds, and if it matters, this belongs in a schema and a validator instead.
• Wrong facts: trace the evidence path before editing wording. Missing or stale evidence may be a retrieval problem, and the source material, the assembly step, the generation itself and your own downstream logic are each candidates.
• Right most of the time, wrong occasionally, in ways that matter: stop editing the prompt. That failure rate is the thing to design around.

The last one is the point at which prompting stops being the right tool. You are not trying to make a probabilistic system stop varying. You are deciding what happens on the runs that vary.

Treat instructions like code

If a prompt is doing real work, it deserves the handling code gets. Keep it in version control rather than pasted in a dozen places. Note what you expected a change to do. Keep a small fixed set of inputs and run them after an edit, because a wording change made for one case routinely breaks another, and without that set you will not notice.

This is unglamorous and it is most of the difference between a prompt that holds up and one that is quietly rewritten every week.

Where this hands off

Once you decide a requirement has to be enforced rather than requested, you have left this page. The next question is what software can actually check: whether the response parses, whether it satisfies an explicit schema, and whether the values make sense.

That is structured output and why it matters, and it carries the boundary this page stops at.