“Neural networks (at least in the configurations that have been dominant over the last three decades) have trouble generalizing beyond the multidimensional space that surrounds their training examples.”
Our reading, the sentence that was checked:
Neural networks in the configurations dominant for the last three decades have trouble generalizing beyond the space surrounding their training examples.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Breaking news: Scaling will never get us to AGI, paragraph 1.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-breaking-news-scaling-will-never-98c7b0-a1
We recorded 6 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2024-04-08. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- in configurations dominant over the last three decades (qualifier, stated by the speaker)
From the speaker
- Neural networks
- configurations that have been dominant over the last three decades
- have trouble generalizing
- beyond the multidimensional space that surrounds their training examples
Our interpretation
- I read 'have trouble' as a recurring difficulty, not an assertion that successful generalization is impossible.
- The ordinary reading is difficulty with materially unfamiliar inputs, relationships, or distributions. A literal boundary around training points is a different mathematical claim.
- The three-decade reference defines the historical scope of the architectures being discussed. It does not require every architecture to fail in every year.
Counts as supported if: Research on commonly used neural architectures finds substantial, recurring performance losses under distribution shifts or extrapolation beyond familiar training conditions.
Counts as contradicted if: Representative controlled evaluations show dependable generalization across materially unfamiliar conditions, undermining the claimed recurring difficulty.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The ordinary claim of recurring difficulty is supported. WILDS finds substantial losses under real-world distribution shifts, and controlled graph-network research identifies extrapolation failures. Successful compositional learning and the geometry research require clear qualifications: neural networks can generalize beyond training examples, and a literal convex-hull boundary does not explain their abilities.
Limitations
- The outcome depends on reading the spatial language as unfamiliar conditions rather than a precise geometric boundary.
- These findings do not establish that scaling can never produce AGI.
Evidence (4 items)
Supports the claim: arXiv; Koh and colleagues,
substantially lower out-of-distribution than in-distribution performance
Why it matters here: WILDS reports performance losses across ten datasets involving natural distribution shifts. This supports recurring generalization difficulty across applications, without establishing a universal inability.
- Where
- Abstract
- Publisher
- arXiv; Koh and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: arXiv; Xu and colleagues,
do not extrapolate well
Why it matters here: The paper identifies a concrete extrapolation failure for a common graph-network configuration. It also shows that suitable architectural choices and training distributions can permit successful extrapolation.
- Publisher
- arXiv; Xu and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv; Balestriero, Pesenti and LeCun,
as an indicator of generalization performances
Why it matters here: The authors show why high-dimensional learning commonly occurs outside the training points' convex hull. This challenges a literal geometric reading, but does not establish reliable performance under distribution shifts.
- Publisher
- arXiv; Balestriero, Pesenti and LeCun
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Nature; Lake and Baroni,
several systematic generalization benchmarks
Why it matters here: A neural network trained for compositional skills demonstrates successful systematic generalization. This limits any universal interpretation, while remaining compatible with difficulty under conventional training.
- Where
- Abstract
- Publisher
- Nature; Lake and Baroni
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Breaking news: Scaling will never get us to AGI
Neural networks (at least in the configurations that have been dominant over the last three decades) have trouble generalizing beyond the multidimensional space that surrounds their training examples.
- Where
- paragraph 1
- Published
- 2024-04-08
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-breaking-news-scaling-will-never-98c7b0-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-breaking-news-scaling-will-never-98c7b0-a1.