“That is in part because you can’t build reliable agents unless you can solve the hallucination problem, and there is still no principled trustworthy answer to that problem.”
Our reading:
Reliable AI agents cannot be built unless the hallucination problem is solved.
Assessment: Unresolved
What happened
Does the evidence support the cause?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The evidence shows how hallucinations can impair agents, but that does not establish that solving hallucination is necessary for reliability. MAKER shows that a system can correct fallible model outputs during a long structured task. Its narrow scope, and the difference between general errors and hallucinations, prevent it from decisively contradicting Marcus in context. Neither side of the necessity claim is established.
Caveats
- Both cited studies are preprints.
- No numerical reliability standard was supplied by the speaker.
- System-level mitigation could itself count as solving the relevant hallucination problem.
- The later studies bear on the necessity relationship, not on a separate historical claim about January 2025.
Evidence (2 items)
Context only: arXiv; Xuannan Liu and colleagues,
degrading overall reliability.
Why it matters: The study describes hallucination propagation as a risk to agent reliability. A risk mechanism does not show that every agent architecture must fail unless hallucination is solved; the event date identifies the reported research finding, not a specified experiment day.
- Publisher
- arXiv; Xuannan Liu and colleagues
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Context only: arXiv; Elliot Meyerson and colleagues,
With these calibrations, the full system solved the problem perfectly.
Why it matters: MAKER reports completing a million-step Towers of Hanoi task using decomposition, voting and rejection of suspect outputs. This challenges an inference from component errors to inevitable system failure, but it is a narrow execution task and does not demonstrate broad reliability with specifically measured unresolved hallucinations.
- Publisher
- arXiv; Elliot Meyerson and colleagues
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
How we judged it
- Kind of claim
- Causal claim
- Deadline
- None given. We don’t add one.
Conditions
- unless hallucination is solved (their words)
Where we had to interpret
- Assess the stated necessity relationship. Do not separately score the accompanying assertion that no trustworthy hallucination solution existed.
- In context, reliable agents means agents that dependably complete varied delegated tasks under ordinary changes in circumstances. The post expressly allows narrow use cases.
- Reliability concerns completed tasks, not necessarily error-free intermediate model outputs. Marcus supplies no numerical reliability threshold.
- Solving the hallucination problem could mean eliminating hallucinations or reliably controlling their effects. A result that depends on choosing between those readings cannot settle the claim.
- The joint presence of hallucinations and unreliable agents does not establish necessity. Discovering a hallucination solution would not contradict this necessity claim.
Right if: Evidence establishing that unresolved hallucinations necessarily prevent dependable agent operation within the stated scope, including when detection, correction and containment are considered. Observed failures alone are insufficient.
Wrong if: Reproducible evidence of dependable agents across varied delegated tasks despite demonstrated unresolved hallucinations, with enough architectural detail to distinguish tolerating hallucinations from solving the operative hallucination problem.
We wrote this test after the outcome was known.
What would settle it
A reproducible, broadly applicable agent evaluation jointly measuring dependable outcomes and remaining hallucinations could supply a counterexample. A necessity result would need to cover plausible correction and containment methods under explicit assumptions.
We found 10 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, AI Agents: Hype versus Reality, redux, paragraph 11.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-01-15.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, AI Agents: Hype versus Reality, redux
That is in part because you can’t build reliable agents unless you can solve the hallucination problem, and there is still no principled trustworthy answer to that problem.
- Where
- paragraph 11
- Published
- 2025-01-15
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.