“I do actually think that these problems will eventually get ironed out, possibly after some important fresh discoveries are made, but whether they get ironed out soon is another matter entirely.”
Our reading, the sentence that was checked:
Marcus thinks chatbot hallucination problems will eventually get ironed out, possibly after some important fresh discoveries are made, but whether they get ironed out soon is another matter entirely.
Assessment: Unresolved No deadline stated on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Oops! How Google bombed, while doing pretty much exactly the same thing as Microsoft did, with similar results, paragraph 19.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-oops-how-google-bombed-while-doing-da8acd-a2
We recorded 8 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-02-09. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “eventually”
- Deadline state when assessed
- No deadline stated
- Deadline state on 2026-10-29
- No deadline stated (the release’s reference date)
Conditions
- possibly after some important fresh discoveries are made (qualifier, stated by the speaker)
- whether soon is another matter entirely (qualifier, stated by the speaker)
From the speaker
- these problems will eventually get ironed out
- possibly after some important fresh discoveries are made
- whether they get ironed out soon is another matter entirely
Our interpretation
- The forecast is that chatbot hallucinations will eventually be fixed, to the point where they are no longer a major reliability problem. 'Possibly after fresh discoveries' is a hedge, not a condition. The 'soon' part makes no commitment.
- No deadline is given, so the forecast cannot be contradicted by time passing.
Counts as supported if: Credible independent evaluations show that leading chatbots no longer hallucinate in any meaningful way across representative real-world uses.
Counts as contradicted if: No contradiction is possible before an end point. 'Eventually' has no deadline.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No deadline stated
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Marcus forecast that hallucination problems will 'eventually' be ironed out. He gave no deadline. Under the contract, the forecast can be scored as supported early only if independent evidence shows hallucinations in leading chatbots are no longer a significant reliability problem across representative uses. The packet has no such evidence. Vectara's leaderboard, released in November 2025, measured hallucination rates above 10% for several leading models, including GPT-5 and Claude Sonnet 4.5. The lowest rate it reported was 3.3% (e1). In September 2025, OpenAI said GPT-5 has fewer hallucinations but 'they still occur', and that hallucinations remain 'a fundamental challenge for all large language models' (e2). Because the forecast has no deadline, the contract sets no way for it to be contradicted.
Limitations
- 'Ironed out' has no stated threshold, so any future judgement would need to set one.
- Benchmark hallucination rates depend on the task and the dataset used.
- The packet's evidence runs only to November 2025 and covers one benchmark and one developer statement.
Evidence (2 items)
Context only: Vectara,
Other notable thinking models like Claude Sonnet 4.5, GPT-5, GPT-OSS-120B, Grok-4, or Deepseek-R1 all have a hallucination rate > 10%.
Why it matters here: Frontier models still hallucinated at measurable rates in late 2025, so no early resolution has happened.
- Publisher
- Vectara
- Kind of source
- Benchmark result
- Describes events of
- Public from
Context only: OpenAI,
Hallucinations remain a fundamental challenge for all large language models, but we are working hard to further reduce them.
Why it matters here: A leading developer says hallucinations were reduced but not solved as of September 2025.
- Where
- paragraph 2
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
Credible independent evaluations showing that leading chatbots no longer hallucinate in any meaningful way across representative real-world uses would support the forecast. Because the forecast has no deadline, the contract provides no evidence that could contradict it.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Oops! How Google bombed, while doing pretty much exactly the same thing as Microsoft did, with similar results
I do actually think that these problems will eventually get ironed out, possibly after some important fresh discoveries are made, but whether they get ironed out soon is another matter entirely.
- Where
- paragraph 19
- Published
- 2023-02-09
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-oops-how-google-bombed-while-doing-da8acd-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-oops-how-google-bombed-while-doing-da8acd-a2.