“The biggest intrinsic failings of generative AI have to do with reliability, in a way that I believe can never be solved, given their inherent nature.”
Our reading, the sentence that was checked:
Generative AI's reliability failings can never be solved, given its inherent nature.
Assessment: Unresolved No deadline stated on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic, paragraph 12.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-alphaproof-alphageometry-chatgpt-cab6c8-a1
We recorded 17 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-07-28. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No deadline stated
- Deadline state on 2026-10-29
- No deadline stated (the release’s reference date)
Conditions
- given their inherent nature (qualifier, stated by the speaker)
From the speaker
- intrinsic failings
- reliability
- can never be solved
- given their inherent nature
Our interpretation
- The plain reading is that generative AI cannot overcome its general reliability problems, not merely that current systems sometimes fail.
- Solved means dependable performance across varied, unfamiliar tasks, with robust error control; it does not require mathematical perfection on every conceivable input.
- Given their inherent nature states the proposed explanation, not a separate event that must happen before scoring.
- Narrow success using an external verifier does not by itself establish that generative AI's general reliability problem is solved.
Counts as supported if: A sound, applicable impossibility result establishes that the relevant class of generative systems cannot achieve the stated general reliability.
Counts as contradicted if: A generative system demonstrably overcomes the relevant reliability failures across broad, independently tested tasks, including unfamiliar inputs and repeated trials.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No deadline stated
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The reviewed evaluations show continuing reliability problems and some improvements. Present failures cannot prove that reliability can never be solved, while partial improvements cannot refute that claim. No decisive general counterexample or applicable impossibility result was established.
Limitations
- The cited empirical study covers two benchmarks and a finite set of agents.
- The distinction between intrinsic model reliability and system-level safeguards remains important.
- No numerical definition of solved was supplied.
Evidence (1 item)
Context only: Princeton University researchers; arXiv,
overall reliability only shows small improvements over time.
Why it matters here: The study documents persistent reliability problems alongside some improvement across the evaluated agents. Neither finding establishes what all future generative systems can or cannot achieve.
- Publisher
- Princeton University researchers; arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Independent, reproducible evidence of broad and robust reliability in a qualifying generative system, or a sound impossibility result covering the relevant systems and tasks.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
The biggest intrinsic failings of generative AI have to do with reliability, in a way that I believe can never be solved, given their inherent nature.
- Where
- paragraph 12
- Published
- 2024-07-28
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-alphaproof-alphageometry-chatgpt-cab6c8-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-alphaproof-alphageometry-chatgpt-cab6c8-a1.