“Many of us, including Melanie Mitchell, Subbarao Kambahapati, Emily Bender, Ernie Davis, etc. have been emphasizing for ages that there are principled limits with LLMs.”
Our reading, the sentence that was checked:
Marcus and many others, including Melanie Mitchell, Subbarao Kambhampati, Emily Bender and Ernie Davis, have long emphasized that there are principled limits with LLMs.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, CONFIRMED: LLMs have indeed reached a point of diminishing returns, paragraph 12.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-confirmed-llms-have-indeed-reached-ce65c9-a1
We recorded 13 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-11-09. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- many of us, including named researchers (scope, stated by the speaker)
From the speaker
- Many of us
- including Melanie Mitchell, Subbarao Kambahapati, Emily Bender, Ernie Davis
- have been emphasizing for ages
- principled limits with LLMs
Our interpretation
- This is a historical attribution: the named people publicly raised limitations well before the post. It does not establish that their arguments were correct.
- Read 'principled limits' as arguments connecting limitations to training, representation, understanding, or reasoning mechanisms, rather than isolated mistakes.
- Read 'for ages' colloquially as a pre-existing body of criticism, not a specified minimum number of years for every person.
- The historical period has no stated beginning and ends immediately before publication.
Counts as supported if: Dated publications and independent contemporary reporting document such arguments across the named researchers before November 9, 2024, including substantial history before 2024.
Counts as contradicted if: Reliable historical records refute the attributions or show that the supposed longstanding arguments only arose contemporaneously with or after the post.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The record supports the ordinary historical reading. Bender, Marcus, and Davis made relevant arguments in 2020; Mitchell discussed mechanism-based reasoning limitations in 2023; and Kambhampati's pre-post paper explicitly argued for planning limitations. This establishes an existing body of criticism. It does not establish that every researcher claimed immutable limits or agreed with Marcus's broader conclusions.
Limitations
- 'For ages' is colloquial, and the evidence does not establish an identical duration for every researcher.
- Mitchell's position is more qualified than a claim of permanent impossibility.
- The outcome depends on reading 'principled limits' as mechanism-based concerns.
- The ACL source supplies a publication month, so its availability date is a conservative month-end bound.
Evidence (4 items)
Supports the claim: Association for Computational Linguistics,
In this position paper, we argue that a system trained only on form has a priori no way to learn meaning.
Why it matters here: Bender and Koller's paper makes an explicit conceptual-limit argument more than four years before the post. The metadata gives July 2020; July 31 is used as a conservative availability bound.
- Where
- Abstract
- Publisher
- Association for Computational Linguistics
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Supports the claim: Melanie Mitchell, AI: A Guide for Thinking Humans,
Whether LLMs have such general abstract-reasoning capacities, elicited through prompting tricks, scratchpads, or other external enhancements, still needs to be systematically demonstrated.
Why it matters here: Mitchell discusses memorization, pattern matching, and failures of generalization more than a year before the post. Her conclusion is qualified: it challenges demonstrated general reasoning rather than claiming a proof of permanent impossibility.
- Where
- Final paragraph
- Publisher
- Melanie Mitchell, AI: A Guide for Thinking Humans
- Kind of source
- Secondary summary
- Describes events of
- Public from
Supports the claim: arXiv,
This further suggests that LLMs are more likely doing approximate retrieval of plans than actual planning.
Why it matters here: Kambhampati and coauthors connect planning failures to approximate retrieval and discuss earlier studies. Their argument concerns unaided LLMs and also describes constructive roles for models paired with external verifiers.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: TechTalks,
“It learns correlations between words, and nothing more.”
Why it matters here: The contemporary report attributes this argument jointly to Marcus and Davis and describes their GPT-3 criticism. It establishes that they publicly advanced the argument years earlier, not that the argument was correct.
- Publisher
- TechTalks
- Kind of source
- News report
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, CONFIRMED: LLMs have indeed reached a point of diminishing returns
Many of us, including Melanie Mitchell, Subbarao Kambahapati, Emily Bender, Ernie Davis, etc. have been emphasizing for ages that there are principled limits with LLMs.
- Where
- paragraph 12
- Published
- 2024-11-09
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-confirmed-llms-have-indeed-reached-ce65c9-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-confirmed-llms-have-indeed-reached-ce65c9-a1.