“But many others have found that LLMs are not quite as advertised.”
Our reading, the sentence that was checked:
Many others have found that LLMs are not quite as good as advertised.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Two takes on AI and the future of America, paragraph 2.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-two-takes-on-ai-and-the-future-of-bc568e-a1
We recorded 8 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-12-23. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- many others have found (scope, stated by the speaker)
- not quite (qualifier, stated by the speaker)
From the speaker
- many others have found
- LLMs are not quite as advertised
Our interpretation
- The ordinary reading is that substantial groups of researchers or users reported disappointing performance or deployment benefits relative to promoted expectations.
- This does not claim that every LLM deployment fails or that a particular advertisement is legally misleading.
- The context describes accumulated findings during 2025, including the July study; use January 1–December 23, 2025.
Counts as supported if: Multiple independent research or user groups report material shortfalls in LLM reliability, productivity or deployment benefits relative to promoted expectations.
Counts as contradicted if: The cited findings are materially misrepresented, or the relevant records establish that the purported shortfalls were not reported.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
On the ordinary reading that many users and organizations reported disappointing results, the claim is supported. The enterprise report documents deployment failures, and the developer survey documents declining confidence in accuracy. These findings coexist with reported productivity benefits. They do not establish that LLMs generally fail, or independently validate every numerical claim in Marcus's surrounding discussion.
Limitations
- The advertising comparison is qualitative and supplied by interpretation.
- NANDA's findings are preliminary and sample-limited.
- Survey perceptions are not controlled productivity measurements.
- The NANDA availability date is a verified latest bound; its exact original publication day was not established.
Evidence (3 items)
Supports the claim: Project NANDA authors; copy hosted by MLQ,
Most fail due to brittle workflows
Why it matters here: The January–June research documents deployment shortfalls across interviewed organizations. August 20 is a verified public-availability bound, not an asserted exact release date for the July-dated report.
- Publisher
- Project NANDA authors; copy hosted by MLQ
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: Stack Overflow,
trust in the accuracy
Why it matters here: This republication summarizes the survey released July 29, before Marcus's post. It documents substantial user dissatisfaction, although declining trust is not a direct test of a specific advertisement.
- Publisher
- Stack Overflow
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: Stack Overflow,
69%
Why it matters here: The same survey also records perceived benefits. This limits any broader claim that users generally find AI useless, but does not negate the existence of many disappointed users.
- Publisher
- Stack Overflow
- Kind of source
- Independent measurement
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Two takes on AI and the future of America
But many others have found that LLMs are not quite as advertised.
- Where
- paragraph 2
- Published
- 2025-12-23
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-two-takes-on-ai-and-the-future-of-bc568e-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-two-takes-on-ai-and-the-future-of-bc568e-a1.