“A quarter century later, hallucinations remain ubiquitous. Within a neural network substrate, the problem still has not been solved.”
Our reading, the sentence that was checked:
Hallucinations remain ubiquitous and unsolved in neural-network-based systems.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI, paragraph 33.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-how-o3-and-grok-4-accidentally-vindicated-a4d0a8-a2
We recorded 47 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-07-13. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- within a neural network substrate (qualifier, stated by the speaker)
From the speaker
- hallucinations remain ubiquitous
- Within a neural network substrate
- the problem still has not been solved
Our interpretation
- In this discussion of LLMs, the ordinary reading is that hallucinations remain a widespread problem across leading neural language models. It does not mean every response or every neural-network application hallucinates.
- Unsolved means no demonstrated general elimination of the problem, rather than no improvement.
- For this ongoing technical observation, use the current calendar year's publicly reported model evaluations through the post date. Technical evaluations have no common quarterly reporting cycle.
Counts as supported if: Evaluations of multiple leading neural language models show recurring hallucinations, including in recently released systems.
Counts as contradicted if: Representative evaluations establish that leading neural language models generally and reliably avoid hallucinations across factual tasks.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
OpenAI's April 16, 2025 system card reports SimpleQA hallucination rates of 51%, 79% and 44% for o3, o4-mini and o1, respectively; PersonQA rates were 33%, 48% and 16% [e1]. Anthropic's May 22, 2025 system card reports hallucination concerns from testing near-final Claude Opus 4 and Sonnet 4 models [e2]. These disclosures support the claim that hallucinations persisted across leading language models before July 13, 2025 and had not been generally eliminated [e1, e2].
Limitations
- Both sources are developer disclosures [e1, e2].
- The benchmark and internal-testing findings do not establish hallucination rates in ordinary use or across all neural-network applications [e1, e2].
- Persistent failures do not establish that neural approaches can never eliminate hallucinations [e1, e2].
Evidence (2 items)
Supports the claim: OpenAI,
more inaccurate/hallucinated claims.
Why it matters here: The release evaluation reports SimpleQA hallucination rates of 51%, 79%, and 44% for o3, o4-mini, and o1, respectively; PersonQA rates are 33%, 48%, and 16%. These are substantial benchmark failure rates across several models, not estimates of everyday usage.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: Anthropic,
sycophancy, hallucination
Why it matters here: Testing of approximately final Claude Opus 4 and Sonnet 4 models also found hallucinations. The card reports successful mitigation of one specific hallucination pattern elsewhere, but says that mitigation did not completely eliminate it.
- Publisher
- Anthropic
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
A quarter century later, hallucinations remain ubiquitous. Within a neural network substrate, the problem still has not been solved.
- Where
- paragraph 33
- Published
- 2025-07-13
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-how-o3-and-grok-4-accidentally-vindicated-a4d0a8-a2: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-how-o3-and-grok-4-accidentally-vindicated-a4d0a8-a2.