“Success on a few problems with a particular strategy in no way guarantees that we can solve all problems in a similar way. It is sheer inductive folly to imagine otherwise, particular when the failure modes (unreliability, bizarre errors, failures in compositionality and discomprehension) have not changed since Fodor and Pinker pointed them out (in separate articles) in 1988.”
Our reading, the sentence that was checked:
Success on a few problems with a particular strategy in no way guarantees that all problems can be solved the same way, and it is inductive folly to think otherwise, especially as the failure modes (unreliability, bizarre errors, failures of compositionality and discomprehension) have not changed since Fodor and Pinker pointed them out in 1988.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, The New Science of Alt Intelligence, paragraph 58.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-the-new-science-of-alt-intelligence-7679e7-a1
We recorded 20 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-05-14. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- Success on a few problems
- in no way guarantees
- all problems in a similar way
- unreliability, bizarre errors, failures in compositionality and discomprehension
- have not changed since Fodor and Pinker pointed them out
- in separate articles
- in 1988
Our interpretation
- The checkable historical assertion is that the broad kinds of failure identified in the 1988 critiques remained present in neural systems by May 14, 2022.
- I read 'have not changed' as persistence of failure categories, rather than identical error rates, unchanged architectures, or no progress on individual problems.
- I read compositionality as applying familiar components in unfamiliar combinations, and discomprehension as observable errors in interpreting meaning.
- The target period follows the explicit comparison from 1988 through the post date. It does not extend to systems introduced after the post.
- The statement about guarantees is a logical caution, and 'inductive folly' is an evaluative judgment. Neither is treated as a separate empirical forecast or as a prediction that neural methods can never succeed.
Counts as supported if: The original critiques identify problems with systematic generalization and language processing, and research available by the post documents analogous compositional, comprehension, and unreliable or anomalous output failures in contemporary neural systems.
Counts as contradicted if: The original articles do not identify the relevant problems, or evidence establishes that the broad failure categories had been overcome by the post date. Improvements on particular tasks alone do not contradict persistence of the categories.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Supported on the ordinary reading that broad failure categories persisted. The 1988 articles raise problems with systematic language processing and generalization. April 2022 research still documents compositional difficulties, semantic-understanding errors, hallucinations, and other unreliable outputs. The stronger reading that nothing relevant improved is not supported: PaLM's report records errors corrected by scaling, and Kirov and Cotterell report overcoming many specific Pinker–Prince objections. These improvements do not establish that the broad categories disappeared. This assessment does not endorse the rhetorical judgment 'inductive folly' or infer that neural approaches can never solve the remaining problems.
Limitations
- The outcome depends on interpreting 'have not changed' as persistence of categories.
- The cited tests cover particular systems and tasks, not every neural architecture.
- The evidence does not establish a common cause for old and modern failures.
- The historical observation ends on May 14, 2022; subsequent capabilities are outside its scope.
Evidence (5 items)
Supports the claim: arXiv; Kensen Shi and colleagues,
the tasks remain challenging.
Why it matters here: Tests involving SCAN and RobustFill found continuing difficulty combining learned components in new ways, even with modified Transformers. This supports persistence of compositional generalization problems while explicitly acknowledging progress.
- Publisher
- arXiv; Kensen Shi and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: arXiv; Aakanksha Chowdhery and colleagues, Google,
hallucinations, repetitive outputs
Why it matters here: The PaLM analysis identifies semantic-understanding errors, omitted reasoning steps, and anomalous outputs; Figure 9 shows that scaling corrected some but not all errors in each category. This supports persistent comprehension and reliability problems, while also providing evidence against unchanged error frequency.
- Publisher
- arXiv; Aakanksha Chowdhery and colleagues, Google
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Cognition; full-text copy hosted by the University of Waterloo,
Several arguments for combinatorial structure in mental representations are then reviewed.
Why it matters here: Fodor and Pylyshyn connect systematic understanding of related sentences to structured representations and challenge connectionist accounts on that basis. This establishes the historical comparison, not an empirical verdict on 2022 systems.
- Publisher
- Cognition; full-text copy hosted by the University of Waterloo
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Context only: Cognition; full-text reproduction hosted by Reinhard Blutner,
for certain words.
Why it matters here: Pinker and Prince document failures in a neural language-learning model, including missing outputs and faulty linguistic generalizations. Their task was past-tense learning, so connecting these failures to modern comprehension errors is a comparison of broad categories.
- Publisher
- Cognition; full-text reproduction hosted by Reinhard Blutner
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Context only: arXiv; Christo Kirov and Ryan Cotterell,
obviates many
Why it matters here: This direct revisit reports substantial improvements on the original past-tense problem and resolution of many specific objections. It challenges a literal claim of unchanged deficiencies, but does not establish elimination of the broad failure categories across neural AI.
- Publisher
- arXiv; Christo Kirov and Ryan Cotterell
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, The New Science of Alt Intelligence
Success on a few problems with a particular strategy in no way guarantees that we can solve all problems in a similar way. It is sheer inductive folly to imagine otherwise, particular when the failure modes (unreliability, bizarre errors, failures in compositionality and discomprehension) have not changed since Fodor and Pinker pointed them out (in separate articles) in 1988.
- Where
- paragraph 58
- Published
- 2022-05-14
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-the-new-science-of-alt-intelligence-7679e7-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-the-new-science-of-alt-intelligence-7679e7-a1.