Human-in-the-Loop Is a System Design Choice

Somewhere in most AI workflows there is a screen with an Approve button. Its presence is usually cited as the reason the system is under control.

Sometimes it is. Often the person pressing it has no way to tell whether they should, no option except pressing it, and no idea what happens if they do not. The button exists. The control does not.

Three different things

Requiring a human signal is not the same as reserving a decision for human judgment. It helps to separate three claims that get made interchangeably:

A person is present. Someone appears somewhere in the workflow. This says nothing on its own; they may be receiving a notification after the fact.

The workflow requires their approval. It will not proceed without a signal from them. This is a real constraint, and it is still only a constraint on sequence.

The person can judge. They have what they need to evaluate the specific thing being asked, and something other than yes is available to them.

Only the third is oversight in any useful sense.

The first two are frequently mistaken for it, usually in good faith, because from an architecture diagram they look identical.

In, on, or out of the loop

Three labels describe where the person sits, and they are often abbreviated as HITL, HOTL and HOOTL.

Human in the loop (HITL): the workflow waits for a person before the step goes ahead. Human on the loop (HOTL): the system acts on its own while a person watches and can step in or stop it. Human out of the loop (HOOTL): no person in the running path; review, if any, happens afterwards.

The labels say where the person is. They do not say whether that person can judge. A human in the loop who cannot say no is closer to a notification than to a decision, which is the test the rest of this page applies.

Is the decision still open?

The first test is timing, and it disqualifies more review steps than anything else.

If the message has been sent, the file deleted, the page published or the payment made, then what a person does next is not approval. It may be monitoring, or audit, or the beginning of a correction, and those are worth having. They are not the same as deciding whether the thing should happen, because it already happened.

So: at the moment the reviewer looks, is the outcome still undetermined? If not, be honest about what the step is. Calling after-the-fact notification an approval gate is how teams end up believing a decision was made that never was.

Which human, and why them

"A human reviews it" is not a design. Humans are not interchangeable, and the useful questions are specific.

Who owns this decision? Not who is available, but whose call it actually is. If nobody can answer that, the review step is a diffusion of responsibility rather than a placement of it.

Does that person have the standing to decide? Assigning review to someone without the relevant authority can produce an approval signal that does not actually settle the decision.

Do they have the knowledge? A reviewer who cannot evaluate the thing in front of them is a delay, not a control.

This is where the old framing of "add a human in the loop" quietly fails. It treats the human as a component type rather than a named person with authority and knowledge.

What the reviewer sees

A button alone is not enough to make a decision, but neither is everything the system knows. The information should be proportionate to what is actually being decided.

For a decision about publishing something, that might be the artifact itself and what changed. For a decision about an exception, it might be the rule, the reason for departing from it, and what the departure costs. What matters is that the reviewer receives what the reserved decision requires, rather than a summary that describes the work without exposing the question.

The interface is part of the control design, not decoration around it, because it determines what evidence and what alternatives exist at the moment of decision. That is a narrower claim than "good UX produces good judgment", which is not something an interface can deliver.

One caution about the surface itself. A review surface that repeatedly presents similar choices with little useful context gives the reviewer little basis for telling one decision from another. That is an argument for reserving fewer decisions and giving each the context its judgment needs, rather than for adding warnings to the ones already there.

What happens when they say no

A review step is weak when refusing has no implemented consequence.

This is the part most often missing. Approval paths get built carefully. Rejection paths get a toast notification.

Refusal needs to be a state the system actually has. Depending on the workflow that might mean the artifact returns for revision with the reviewer's reason attached, the run stops and releases what it was holding, more evidence is requested, the decision escalates to a different owner, or the work falls back to a manual route.

Not every workflow needs all of those. If a workflow asks a human to decide, at least one meaningful non-approval transition must exist, because if the only thing refusal does is prevent the next step until someone approves anyway, the reviewer does not have a decision. They have a delay they are responsible for.

A worked example, and what it does not show

By default the publishing script prepares a branch, pushes it and opens a pull request without merging it, and when a flag is supplied it can instead arm GitHub to squash-merge that pull request automatically once the required conditions pass. Validation runs first either way: frontmatter, body structure, the editorial contract and the prose rules, then the collection-wide checks. If those fail after the content has been placed, the script restores the previous state rather than leaving it half-applied.

What that reserves is specific. Everything up to an open pull request is automated. The decision left open is whether this becomes the live site. A successful continuous integration run on the main branch triggers the deploy workflow that publishes the site, so on the default path the merge is that decision.

Two things this example does not establish.

The script also accepts a flag that arms auto-merge, which squash-merges as soon as the checks pass. On that path nobody merges by hand.

That difference is worth being exact about, because the repository supports only half of it. On the default path, the merge can itself be the human publication decision. On the auto-merge path, the script's own comments say approval happened earlier at a review gate, and this repository does not record who or what supplied it. The mechanism assumes a prior authorization. The code is not evidence that one occurred.

What the example does show is narrower and still useful. Automation can continue after a human decision without removing that decision from the control path. Whether a person performs the mechanical step is a different question from whether a decision was reserved for one. This repository demonstrates the automation path. It does not establish that the assumed earlier human decision occurred.

And the validation described above is machine validation. It checks rules that were written down. It does not decide whether the piece is worth publishing, and the script passing says nothing about that.

Validation answers a different question

Machine checks establish that explicit rules hold. Passing them does not answer whether this should be published, whether this exception should be accepted, whether the interpretation is right, or whether the evidence justifies the decision.

Those are questions someone chose to reserve. That choice is the design act, and it is separate from whether an automated check can be written. A rule that can be checked usually should be, precisely so the reserved decisions are the ones that genuinely need a person.

This is also why an approval is not a readiness verdict. Someone approving one artifact has not certified that the system is fit for real users, which is a different question with its own page.

Not a universal fallback

"When the system is unsure, ask a human" fails as a general rule. Review costs time, needs capacity, and needs the right person available. A workflow that escalates everything uncertain becomes a queue, and queues get approved quickly.

The goal is not more human involvement. It is deliberate ownership: deciding which decisions stay with a person, and then making those decisions genuinely possible to make.

What this changes

Take an approval step you already have and ask five things. What decision is actually stopping here? Who owns it? What does the reviewer see? What can they do other than approve? What happens when they decline?

If any answer is missing, the gate is doing less than it appears to.

Whether the action was permitted in the first place is a different question, handled in governed execution.