“At the same time, more optimistic predictions like these [from Reid Hoffman September 7, 2023] have not thus far borne out.”
Our reading, the sentence that was checked:
Reid Hoffman's optimistic September 2023 AI predictions had not come true as of March 2024.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, When Will the GenAI Bubble Burst?, paragraph 8.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-when-will-the-genai-bubble-burst-27395f-a2
We recorded 19 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-03-31. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- thus far (qualifier, stated by the speaker)
From the speaker
- more optimistic predictions like these
- Reid Hoffman September 7, 2023
- have not thus far borne out
Our interpretation
- The referenced TIME interview contains Hoffman's prediction that hallucination rates could reach human-expert rates within months.
- The following paragraph about hallucinations makes that prediction the clearest identifiable referent. The supplied text does not establish every prediction contained in the embedded material.
- I assess whether that capability had materialized by March 31, 2024. I do not treat the statement as saying that every possible interpretation of Hoffman's deadline had expired.
- Human-expert parity means comparable factual reliability on representative tasks under comparable conditions, rather than merely passing an examination or improving over an earlier model.
Counts as supported if: Evidence about systems available by March 31, 2024 shows that the referenced human-expert hallucination-rate milestone had not been reached.
Counts as contradicted if: A credible comparison establishes that a system available by March 31, 2024 had reached the referenced human-expert hallucination rate across the relevant factual tasks.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
In his September 7, 2023 interview, Hoffman predicted human-expert hallucination rates “within months.” [e4] Research revised on February 21, 2024 found hallucinations in most evaluated ChatGPT and Llama2-Chat outputs in information-seeking scenarios, but did not compare their error rates with those of experts answering equivalent questions. [e5] These sources do not establish whether systems available by March 31, 2024 had reached human-expert factual reliability. [e4, e5]
Limitations
- The interview identifies the hallucination prediction but does not establish the complete set of predictions Marcus referenced. [e4]
- Human judgments of model outputs do not supply a matched human-expert error rate. The reported hallucinations therefore do not settle the parity question. [e5]
Evidence (2 items)
Context only: TIME,
a solvable problem
Why it matters here: In the same answer, Hoffman predicts human-expert hallucination rates within months. This identifies the relevant prediction but provides no measurement of whether it happened.
- Publisher
- TIME
- Kind of source
- News report
- Describes events of
- Public from
Context only: arXiv; Mishra and colleagues,
factual errors
Why it matters here: The contemporaneous study reports widespread hallucinations in the evaluated outputs and improvements from a correction system. It does not measure human experts producing equivalent answers, so it cannot establish the requested parity comparison.
- Publisher
- arXiv; Mishra and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
A credible comparison of systems available by March 31, 2024 with appropriate human experts on representative factual tasks, matching source access, answer coverage and scoring. The complete referenced material would clarify whether additional predictions require assessment.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, When Will the GenAI Bubble Burst?
At the same time, more optimistic predictions like these [from Reid Hoffman September 7, 2023] have not thus far borne out.
- Where
- paragraph 8
- Published
- 2024-03-31
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-when-will-the-genai-bubble-burst-27395f-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-when-will-the-genai-bubble-burst-27395f-a2.