Why "Human in the Loop" Alone Is Not a Governance Strategy

"Human in the loop" gets treated in most boardrooms as a solved problem — add a person to sign off, and the AI system is governed. Phaedra Boinodiris and Jamie Mackenzie of IBM Consulting argue this confidence is misplaced. Their core claim: a human's presence in the process isn't the same thing as meaningful oversight, and treating it as such lets accountability quietly slip away from where it actually belongs.

What makes any oversight trustworthy in the first place

The authors start from an unusual angle: how do we decide whether to trust a colleague's work? They point to five familiar qualities — is the work accurate (credibility), consistent (reliability), done with the right judgment (dependability), aligned with the team's interests rather than self-interest, and stable over time even as conditions change (integrity). Their argument is that AI systems deserve to be held to these same standards, translated into AI-specific vocabulary: transparency (was this built the right way), explainability (why did it produce this output for this person), observability (is it still behaving correctly as conditions shift), alignment (what was it actually optimized for, and for whom), and robustness (has it been tampered with or quietly drifted since it was last checked).

"Liability laundering"

The piece's sharpest concept: when an AI system errs and no real interrogation mechanism was built in, organizations default to saying "a human reviewed it" — which the authors call liability laundering. Accountability that should sit with the people who designed and deployed the system gets quietly redirected onto whoever clicked "approve," even though that person may have had no real ability to catch the problem.

Four practical failure points

The authors identify four specific ways "human in the loop" breaks down in practice:

• Automation bias isn't a personal failing — it's a predictable response to systems that arrive fast, confident, and without real evidence to interrogate. The fix isn't asking people to be more suspicious; it's giving them something real to be suspicious of.
• Organizations measure presence, not engagement — tracking whether a human touched a decision, not whether they pushed back, asked questions, or had the information needed to disagree. When speed is rewarded and scrutiny is only nominal, speed wins every time.
• Authority is often left undefined — if a reviewer can flag a decision but the system still decides, that's not oversight, it's a performance of oversight. The authors argue every human-in-the-loop role needs explicit answers to what the person can actually change and what happens when they disagree.
• Disagreement rarely produces action — most systems log an override without a named owner obligated to act on the pattern, or any way to verify a fix actually happened. Without that, a "feedback loop" is just record-keeping, and the reviewer learns their judgment doesn't actually matter.

What humans actually bring that models don't

The piece pushes against a narrow view of what "training reviewers" should mean. Rather than turning reviewers into AI engineers, the authors argue the real value humans bring is understanding people — reading a case file against a person's actual circumstances, noticing what's missing from a dataset, grasping what a decision means in someone's life. They frame this as a moment where humanities-trained expertise (social work, ethics, counseling, education) becomes central to governance, not a soft addition to it.

The standard that actually matters

Their closing framing draws a clear line: a human in the loop is not the same thing as a human who is accountable. Most organizations, in their view, are still designing for the former — presence without real power, oversight without real evidence. The alternative requires giving reviewers actual documentation to check, decision-level explanations they can demand, and visibility into when a system has changed since it was last validated.