“That’s exactly what classical AI has been trying to do for decades, with only limited success. And what systems like Gemini still struggle with.”
Our reading, the sentence that was checked:
Classical AI has had only limited success in turning input into checkable logical form and reasoning over it, and systems like Gemini still struggle with it.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, No, RAG is probably not going to rescue the current situation, paragraph 29.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-no-rag-is-probably-not-going-to-rescue-06d564-a1
We recorded 11 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-02-23. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- classical AI
- for decades
- only limited success
- systems like Gemini still struggle
Our interpretation
- That refers to interpreting natural-language and visual inputs into usable representations and reasoning over them.
- In the surrounding discussion of reliably addressing all queries, limited success means substantial bounded achievements without broadly reliable general coverage.
- The historical component covers earlier classical systems; no precise starting year is supplied.
- The Gemini component concerns capabilities publicly documented before February 23, 2024, using the latest reporting available for that period rather than later model releases.
Counts as supported if: Historical research documents domain or generalization limits in input interpretation and formal reasoning, and contemporary Gemini evaluations document relevant reasoning or interpretation failures.
Counts as contradicted if: Evidence establishes broadly reliable interpretation and reasoning across the claimed input scope in historical classical systems or contemporary Gemini. Success confined to specialist domains does not by itself contradict the contextual reading.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Historical research documents domain restrictions in classical semantic parsing, and independent contemporary tests identify specific Gemini Pro reasoning failures. Those findings support the contextual claim of incomplete general reliability. Classical systems and Gemini also had substantial achievements, documented here as counterweight to any reading that dismisses their progress.
Limitations
- Limited success is a qualitative judgment about general coverage.
- The historical synthesis is supported by a primary bounded demonstration, but is not an exhaustive survey of classical AI.
- The independent Gemini findings concern Pro; Google's Ultra results show stronger capabilities.
- Reasoning failures do not establish exactly which internal processing stage failed.
Evidence (4 items)
Supports the claim: arXiv; Kamath and Das,
Early semantic parsing systems were mainly rule-based, making them domain specific.
Why it matters here: The historical synthesis describes domain restrictions and adaptation problems, including the 1984 SAVVY system. This supports limited general coverage, rather than an absence of successful applications.
- Publisher
- arXiv; Kamath and Das
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: arXiv; Akter and colleagues,
‘tracking_shuffled_objects’ tasks.
Why it matters here: Independent tests document failures in tracking objects through natural-language descriptions and reduced performance on longer reasoning questions. These are concrete limitations relevant to the claim, although they do not isolate a formal-representation stage.
- Publisher
- arXiv; Akter and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: MIT Artificial Intelligence Laboratory; Terry Winograd,
answers questions
Why it matters here: The original report demonstrates meaningful language understanding, planning, and deduction in a modeled world of toy objects. It prevents interpreting limited success as no substantial classical achievements.
- Publisher
- MIT Artificial Intelligence Laboratory; Terry Winograd
- Kind of source
- Demonstration
- Describes events of
- Public from
Context only: Google, Gemini Team,
multimodal reasoning tasks.
Why it matters here: Google reports 62.4% on MMMU and demonstrates interpreting a handwritten physics solution. This establishes meaningful multimodal progress while leaving substantial room for failure; it does not contradict the contextual claim of incomplete general reliability.
- Publisher
- Google, Gemini Team
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, No, RAG is probably not going to rescue the current situation
That’s exactly what classical AI has been trying to do for decades, with only limited success. And what systems like Gemini still struggle with.
- Where
- paragraph 29
- Published
- 2024-02-23
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-no-rag-is-probably-not-going-to-rescue-06d564-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-no-rag-is-probably-not-going-to-rescue-06d564-a1.