“Deep learning has undeniably made amazing progress, on other dimensions, but the fact is it hasn’t solved the core problems that many of us thought were essential 30 years ago.”
Our reading, the sentence that was checked:
Deep learning has made progress on other dimensions but has not solved the core problems thought essential 30 years ago, so in that sense it has hit a wall.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Happy Groundhog Day, The AI Edition, paragraph 6.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-happy-groundhog-day-the-ai-edition-40a16a-a2
We recorded 12 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-02-03. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- made amazing progress, on other dimensions
- hasn’t solved the core problems
- essential 30 years ago
- abstraction, compositionality, and the type-token distinction
Our interpretation
- The context identifies the core problems as abstraction, compositionality, and separating knowledge about individual instances from knowledge about kinds.
- Solved means a dependable general capability, not perfect performance on every possible input.
- The assessment concerns the stated set of problems. Evidence about compositionality alone does not establish every named component.
- Hit a wall is assessed only in the limited sense Marcus supplied: important problems remain unresolved despite gains elsewhere. It is not treated as a prediction that future progress is impossible.
- Use the latest completed research year, 2022, and the portion of 2023 preceding the post for the ongoing state.
Counts as supported if: Contemporary evidence establishes substantial progress in generation or language interaction while controlled evaluations establish that the named core capabilities, including individual-versus-kind distinctions, remain unreliable.
Counts as contradicted if: Broad, reproducible contemporary evidence establishes dependable solutions to the named core capabilities, or the asserted progress elsewhere is absent.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The 2022 Imagen and InstructGPT evaluations document progress in image generation and instruction following [e2, e4]. A 2022 semantic-parsing study found persistent compositional generalization limitations, alongside gains from some methods [e1]. These findings support progress alongside remaining compositional problems [e1, e2, e4]. However, the entity-tracking study does not directly test separation of individual facts from category knowledge [e3]. The supplied evaluations therefore do not settle the full set of capabilities required by the claim [e1–e4].
Limitations
- The packet lacks direct evaluations establishing the separate abstraction and individual-versus-kind components.
- Entity-tracking failures do not establish the proposed explanation for hallucinations [e3].
- These evaluations do not establish permanent architectural limits [e1–e4].
Evidence (4 items)
Supports the claim: arXiv; Linlu Qiu and colleagues,
flat or negative
Why it matters here: The study finds persistent compositional generalization limitations across model scales and training methods. It also reports gains from in-context learning and prompt tuning, so it does not establish an absence of progress on the core problems.
- Where
- Abstract
- Publisher
- arXiv; Linlu Qiu and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: arXiv; Chitwan Saharia and colleagues,
photorealism
Why it matters here: The paper reports improved image-generation results, including comparative human evaluation. It supports progress in another dimension, but the authors' description of language understanding is not proof that Marcus's core problems were solved.
- Publisher
- arXiv; Chitwan Saharia and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: arXiv; Long Ouyang and colleagues,
improvements
Why it matters here: The study documents improvements from human-feedback training, including human preference for instruction-following outputs. It supports progress in language interaction while acknowledging remaining mistakes.
- Where
- Abstract
- Publisher
- arXiv; Long Ouyang and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv; Sebastian Schuster and Tal Linzen,
tracking entities.
Why it matters here: GPT-2 and GPT-3 showed unreliable discourse-entity tracking, alongside improvements in larger models. Tracking whether a noun phrase introduces an entity is related to reference but does not directly establish confusion between individual knowledge and category knowledge.
- Publisher
- arXiv; Sebastian Schuster and Tal Linzen
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Controlled evaluations available before the February 3, 2023 post that directly test abstraction and separation of individual facts from category knowledge, alongside compositional generalization, using explicit criteria for dependable performance.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Happy Groundhog Day, The AI Edition
Deep learning has undeniably made amazing progress, on other dimensions, but the fact is it hasn’t solved the core problems that many of us thought were essential 30 years ago.
- Where
- paragraph 6
- Published
- 2023-02-03
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-happy-groundhog-day-the-ai-edition-40a16a-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-happy-groundhog-day-the-ai-edition-40a16a-a2.