Verification Theater: Why Agents Can't Check Their Own Work

Last week we put a price tag on the agent-validation gap: $450 in API credits to brute-force a PCB design that had no verification step at all. Generation works. Engineering correctness doesn't.

This week the picture got sharper — and more uncomfortable. A systematic evaluation published this month shows the problem isn't just that verification is missing. It's that when agents are instructed to verify, they perform verification rather than doing it. Dan Luu calls the pattern out plainly:

"agents tend to either just write the tests they would normally write, but inside a framework for a different type of test technique, or they'll use a technique superficially but not really do the things that get the value out of the technique."

The eval nobody's agents passed

Luu, an independent performance and evaluation practitioner, ran 26 testing and verification prompt conditions against agents implementing Zstd in Rust — 80 runs per condition. TDD, formal methods (Lean 4, TLA+, Alloy, Verus), property-based testing, "make no mistakes," four agent skills. No condition dramatically outperformed default behavior.

The failure mode was behavioral, not capability. With formal methods, agents "proved irrelevant properties" or "wrote vacuous proofs that were effectively A => A." With property-based testing, they ran "low-value random cases against a trivial property." Agents repeatedly failed to reverse bitstream order and wrote palindromic tests that couldn't detect the failure. The headline result: agents don't know how to use these tools or techniques very well — "agents are really bad at testing and don't seem to understand how to test reasonably 'by default.'"

And here's the one that should end the debate about self-verification: the Hegel formal-methods skill made correctness worse while raising cost 26–41%. An agent armed with a verification technique, spending real money, producing less correct output. The verification step was theater — expensive theater.

Luu's own conclusion is the wedge, stated as a question: he explicitly wonders why AI labs haven't built RL environments for teaching agents to test well. Until someone does, asking an agent to check its own work is not a trust primitive.

The failure that passes every eval

The second signal comes from a production multi-agent story published ~September 7 (Towards Data Science): a three-node triage pipeline where the account-history node receives a billing API 200 with an empty payload — the account ID was mangled upstream, and the service quietly returned nothing. The node passes it downstream as valid. The drafting node writes a polite "no billing history" refund denial that ships to the customer.

Every output-level eval — rubrics, LLM judges, human skim — passes. Because the failure wasn't in the output. It was in the handoff:

"The account-history node didn't fail loudly; it failed by succeeding at returning the wrong thing, and succeeding is exactly what output-level eval is built to reward."

"Your final output can lie to you. The trajectory can't."

The author cites Datadog's 2026 State of AI Engineering: ~5% production failure rate for AI requests, and only ~60% of failures are loud. The proposed fix is "Intermediate State Eval" — lightweight watchdog graders between agent nodes, each asking one narrow binary question: does this handoff look sane? Plus explicit handoff schemas.

The positioning writes itself: grade the handoff, not the output. Output-level evaluation rewards plausible failure. Verification has to sit in the path between agent steps, checking state — not style.

Hardware already knows the answer

The third signal is from the physical world, and it's the pattern the industry is converging on. Erqos, an Athens-based industrial IoT controls vendor, added native hardware-in-the-loop support to its EQSP32 microPLC (sub-$200 controller) so an AI agent proves generated control code on real silicon before it touches a plant. Authorship and verification are separated: the agent writes the control program, injects sensor states and faults over USB, observes real outputs against a defined test plan, fixes failures, and repeats until every check passes. Their worked example: functional spec, a 31-case HIL test plan, control program, harness, and test report — 18 minutes unattended, all 31 tests passed.

"The agent has to prove the behaviour of its code on real hardware against a defined test plan, rather than simply generating the code and assuming it is correct."

"A hallucinated setpoint, an invented register, or an off-by-one debounce, fails the hardware check in exactly the way a human typo would."

Co-founder Cosma Pabouctsidis: "The AI writes the control logic, but the hardware decides if it is correct."

Note what HIL catches and what it doesn't. HIL catches hallucinations against a test plan — but not an error in the specification itself. Erqos says so explicitly: that's what the human review gate is for. Verification is layered: deterministic environment for the behavior, a human for the intent.

The circuit breaker is the architecture

The fourth signal generalizes the pattern beyond hardware. Sayali Patil — six-plus years building infra automation at Cisco and Splunk, writing in VentureBeat — describes a remediation agent restarting a latency-spiking microservice. Reasonable action given its training data and its narrow view of the incident. Except it didn't know three other services were at peak, the connection pool was at 87%, and a DB index rebuild was running. The restart triggered a thundering herd. The incident got logged as service restart/latency — "the agent is invisible in the postmortem," because no incident classification captures agent action as initiating cause. (AI Incidents Database: reported AI incidents up 21% from 2024 to 2025.)

"The agent initiated an action. The action was technically correct given the agent's context. The context was incomplete. The infrastructure cascaded."

Patil's prescription: "A circuit breaker that hands ambiguous cases to a human is not a weakness in the agent architecture. It is the thing that makes the architecture trustworthy enough to actually run in production."

An approval gate is not a limitation on the agent. It is the trust primitive that lets the agent run at all.

Four signals, one architecture

These are four independent sources in two weeks:

Signal	What the agent did	What self-verification missed
Luu's 26-condition eval	Applied verification techniques when instructed	Only performed them — vacuous proofs, worse correctness at higher cost
The intermediate-state 200	Passed a poisoned payload downstream	Output evals all passed; the failure was in the handoff
Erqos HIL	Generated control code	Spec-level errors — caught only by the human review gate
The chaos-injector restart	Acted correctly on incomplete context	Nobody's eval tested that combination; the postmortem couldn't see the agent

The convergence is the point. Self-checking agents are not a credible trust primitive — verified behaviorally (Luu), architecturally (intermediate-state eval), physically (HIL), and operationally (the circuit breaker). The market is independently discovering the same three-part shape: external verification (the hardware, the watchdog grader — something that isn't the model checking its own homework), deterministic checks (test plans, handoff schemas, binary sane-or-not questions), and a human gate for intent (spec review, ambiguous-case escalation).

Verification is the merge gate

This is the sequel to the $450 story. That post showed generation works and verification is missing — the economic gap. This one shows the behavior gap: even when instructed to verify, agents perform verification theater. So the verification layer can't be another agent, another prompt, another "check your work" instruction. It has to be structural.

ProductFlo's position, unchanged and now better-evidenced: verification is the merge gate for agent-driven hardware. Agents propose diffs. An external, deterministic evaluation layer grades the handoff — not just the output — against engineering constraints. Humans approve the change. That's the layer that separates authorship from verification, the Erqos pattern, the intermediate-state pattern, the circuit-breaker pattern: one architecture, four witnesses.

Stop bad hardware changes before they ship. Not with a smarter agent — with a gate the agent can't talk its way past.

---

Pilot: a fixed $8,000 / 30-day pilot with a written scorecard — baseline in week one, measured at day 30. Team from $48,000/year. Book a demo.

Sources: Dan Luu, "agentic testing" (September 2026); Towards Data Science, "Why most multi-agent systems fail even when evaluation passes" (~2026-09-07); Drives&Controls, "AI agent proves code on a sub-$200 controller before it reaches a plant" (2026-09-09); VentureBeat, Sayali Patil on agents as chaos injectors (~2026-09-16).