“All that we will have this year are demos.”
Our reading, the sentence that was checked:
In 2025, AI agents will exist only as demos.
Assessment: Contradicted Deadline passed on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, AI Agents: Hype versus Reality, redux, paragraph 5.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-ai-agents-hype-versus-reality-redux-5a7e45-a3
We recorded 6 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-01-15. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “this year”
- Deadline state when assessed
- Deadline passed
- Deadline state on 2026-10-29
- Deadline passed (the release’s reference date)
From the speaker
- All that we will have this year are demos
- this year (2025)
- And those demos won't work all that reliably
Our interpretation
- Plain reading: in 2025, AI agents would exist only as demonstrations, not as products in real paid or production use.
- The previous sentence allows 'narrow use cases' for doing a huge fraction of work. This sentence makes no such exception, but a reader could take 'demos' to mean only general-purpose agents.
Counts as supported if: Through 2025, AI agent products stayed at the demo, preview or experimental stage, with no significant production use or paying customers.
Counts as contradicted if: During 2025, AI agent products were generally available and in real production use with paying customers at meaningful scale.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Contradicted
The dated evidence goes against what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- Deadline passed
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Marcus wrote on 15 January 2025 that all we would have that year were agent demos. The window ended on 31 December 2025. Under the contract's plain reading, the sentence covers agent products in general. It does not carry the 'narrow use cases' exception from the sentence before it. Anthropic says Claude Code became generally available in May 2025 and became a critical tool for firms including Netflix, Spotify and KPMG. Anthropic also says it reached $1 billion in run-rate revenue in November 2025. Salesforce reported more than 9,500 paid Agentforce deals as of the quarter ending 31 October 2025, and a 70% quarterly rise in Agentforce accounts in production. These disclosures show paid production use in 2025, which meets the contradiction condition. A benchmark paper from 30 October 2025 found that the best agent automated 2.5% of the real-world projects in its Remote Labor Index. That bears on his point about reliability. It does not show that agent products were only demos.
Limitations
- The result depends on reading 'demos' as covering all AI agent products, not only general-purpose ones.
- Coding agents and customer-service agents might count as the 'narrow use cases' he allowed in the previous sentence.
- Run-rate revenue and deal counts are company-reported figures, not audited annual revenue.
- The benchmark measures performance on a fixed set of projects, not reliability in deployed products.
Evidence (4 items)
Goes against the claim: Anthropic,
Since becoming generally available in May 2025, Claude Code has grown from its origins as an internal engineering experiment into a critical tool for many of the world’s category-leading enterprises, including Netflix, Spotify, KPMG, L’Oreal, and Salesforce
Why it matters here: An agentic coding product was generally available from May 2025 and used by major companies. That is more than a demo.
- Where
- paragraph 4
- Publisher
- Anthropic
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Goes against the claim: Anthropic,
In November, Claude Code achieved a significant milestone: just six months after becoming available to the public, it reached $1 billion in run-rate revenue.
Why it matters here: A $1 billion revenue run rate within 2025 shows large-scale paid use of an agent product. This is a run rate, not realized annual revenue. It is cited only as evidence of real use.
- Where
- paragraph 1
- Publisher
- Anthropic
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Goes against the claim: Salesforce,
Agentforce accounts in production increased 70% quarter-over-quarter
Why it matters here: An enterprise agent product had more than 9,500 paid deals and growing production accounts in the quarter ending October 2025.
- Publisher
- Salesforce
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: arXiv (Scale AI / Center for AI Safety),
AI agents perform near the floor on RLI, with the highest-performing agent achieving an automation rate of 2.5%.
Why it matters here: General-purpose agents were still unreliable at end-to-end work in 2025. This backs his reliability point, but not the 'only demos' claim.
- Where
- Abstract
- Publisher
- arXiv (Scale AI / Center for AI Safety)
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, AI Agents: Hype versus Reality, redux
All that we will have this year are demos.
- Where
- paragraph 5
- Published
- 2025-01-15
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-ai-agents-hype-versus-reality-redux-5a7e45-a3: Current. Outcome: Contradicted. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-ai-agents-hype-versus-reality-redux-5a7e45-a3.