“If hallucinations were brought down to human expert levels by the end of 2024 I would be truly astonished.”
Our reading, the sentence that was checked:
By the end of 2024, hallucinations by large language models will not have been reduced to the level of human experts.
Assessment: Can’t be checked Deadline passed on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, When Will the GenAI Bubble Burst?, paragraph after the quoted Reid Hoffman predictions.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment gm-hallucinations-expert-level-2024-a4
We recorded 19 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-03-31. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “by the end of 2024”
- Deadline state when assessed
- Deadline passed
- Deadline state on 2026-10-29
- Deadline passed (the release’s reference date)
From the speaker
- 'If hallucinations were brought down to human expert levels by the end of 2024 I would be truly astonished'
- the stated reason: hallucinations 'are inherent to how the system makes predictions'
Our interpretation
- We read 'truly astonished' as a firm expectation that the threshold would not be reached by the end of 2024.
- The speaker named no task, domain, or measure for 'human expert levels'. A test would need us to choose the tasks, the experts, and the definition of a hallucination, and that choice would decide the result.
Counts as supported if: Not scored. Testing the forecast would need a task, domain, and measure for 'human expert levels' that the speaker did not supply.
Counts as contradicted if: Not scored. Testing the forecast would need a task, domain, and measure for 'human expert levels' that the speaker did not supply.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Can’t be checked
The statement is a recommendation, a value judgment, or too open to test. It is recorded, not scored.
- Evidence as of
- Deadline state when assessed
- Deadline passed
- Assessment recorded
- Policy version
- 2026-10-28.5 (the current policy is 2026-10-28.7; this assessment has not been rechecked under it)
Rationale
On March 31, 2024, Marcus wrote that he would be truly astonished if hallucinations were brought down to human expert levels by the end of 2024. That is a firm expectation, but he named no task, domain, or measure for 'human expert levels'. Any baseline we chose would decide the result, so the forecast is recorded and not scored.
Limitations
- This record does not show how often AI systems or human experts made factual errors in 2024.
Evidence (0 items)
No evidence item settled this claim by the cutoff.
What would settle it
Not scored as stated. A restatement naming the tasks and the expert comparison would be scorable.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, When Will the GenAI Bubble Burst?
If hallucinations were brought down to human expert levels by the end of 2024 I would be truly astonished.
- Published
- 2024-03-31
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment gm-hallucinations-expert-level-2024-a4: Current. Outcome: Can’t be checked. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Site correctionBy this site
A new assessment with the same outcome replaced the previous one.
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment gm-hallucinations-expert-level-2024-a3, which was not published.