“AI “Agents” will be endlessly hyped throughout 2025 but far from reliable, except possibly in very narrow use cases.”
Our reading, the sentence that was checked:
AI agents will be heavily hyped in 2025 but far from reliable except possibly in very narrow use cases.
Assessment: Supported Deadline passed on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, 25 AI Predictions for 2025, from Marcus on AI, paragraph 32.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-25-ai-predictions-for-2025-from-marcus-6d352a-a2
We recorded 37 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-01-01. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “throughout 2025”
- Deadline state when assessed
- Deadline passed
- Deadline state on 2026-10-29
- Deadline passed (the release’s reference date)
Conditions
- except possibly very narrow use cases (qualifier, stated by the speaker)
From the speaker
- AI “Agents” will be endlessly hyped throughout 2025
- but far from reliable
- except possibly in very narrow use cases
Our interpretation
- The claim has two parts. (1) Hype: prominent industry figures and vendors heavily promote AI agents during 2025. (2) Reliability: on general multi-step tasks, agents complete well under most tasks or are judged unreliable by credible evaluators. Success in some narrow areas is allowed by his carve-out.
- Whether coding agents count as a 'very narrow use case' is open. Here they are treated as a specialised domain, not as general-purpose agents.
Counts as supported if: Evidence of heavy agent promotion in 2025, plus independent 2025 evaluations or credible experts showing that general-purpose agents fail a large share of realistic tasks.
Counts as contradicted if: Credible 2025 evidence that agents worked reliably across broad, varied real-world tasks, for example high completion rates on realistic workplace benchmarks, or wide dependable deployment beyond narrow niches. Alternatively, evidence that agents were not heavily hyped.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- Deadline passed
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Both parts held in 2025. On hype: Sam Altman opened the year saying agents might 'join the workforce', and Gartner said most agent projects were 'mostly driven by hype'. On reliability: CMU's TheAgentCompany benchmark found the best agent finished only 30% of realistic workplace tasks. Gartner judged that current models lack the maturity to achieve complex goals on their own. In October, Andrej Karpathy said such agents 'just don't work'. The main counterpoint is coding agents, which worked well by some practitioners' accounts. The verdict treats coding as the kind of specialised use Marcus's carve-out allows.
Limitations
- The result partly depends on reading coding agents as a 'narrow use case'. A reader who treats coding as broad could call this mixed.
- 'Endlessly hyped' and 'far from reliable' set no thresholds.
Evidence (5 items)
Supports the claim: Sam Altman (OpenAI CEO) blog,
We believe that, in 2025, we may see the first AI agents “join the workforce” and materially change the output of companies.
Why it matters here: The OpenAI CEO opened 2025 by saying agents might 'join the workforce'. This is an example of the hype the claim predicted.
- Publisher
- Sam Altman (OpenAI CEO) blog
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: Reuters (via The Star),
current models do not have the maturity and agency to autonomously achieve complex business goals or follow nuanced instructions over time
Why it matters here: In mid-2025, Gartner judged that current models lack the maturity to achieve complex goals on their own. The same report says most projects are 'mostly driven by hype' and flags 'agent washing'.
- Publisher
- Reuters (via The Star)
- Kind of source
- News report
- Describes events of
- Public from
Supports the claim: Carnegie Mellon University researchers (arXiv),
even with the strongest frontier model, it only manages to complete 30% of the total tasks
Why it matters here: On 175 realistic workplace tasks, the best 2025 agent (Gemini 2.5 Pro) completed only 30%. That fits 'far from reliable' for general work.
- Publisher
- Carnegie Mellon University researchers (arXiv)
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: Simon Willison's Weblog (quoting Andrej Karpathy on Dwarkesh Podcast),
The reason you don’t do it today is because they just don’t work.
Why it matters here: In October 2025, OpenAI co-founder Andrej Karpathy said employee-like agents 'just don't work' and called 'the year of agents' more likely a decade.
- Publisher
- Simon Willison's Weblog (quoting Andrej Karpathy on Dwarkesh Podcast)
- Kind of source
- Secondary summary
- Describes events of
- Public from
Goes against the claim: Simon Willison's Weblog,
Coding agents that are really good at running tools in the loop on the other hand are here already.
Why it matters here: A practitioner says coding agents worked well in 2025. This counts against the claim only if coding falls outside Marcus's 'very narrow use cases' carve-out.
- Publisher
- Simon Willison's Weblog
- Kind of source
- Secondary summary
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, 25 AI Predictions for 2025, from Marcus on AI
AI “Agents” will be endlessly hyped throughout 2025 but far from reliable, except possibly in very narrow use cases.
- Where
- paragraph 32
- Published
- 2025-01-01
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-25-ai-predictions-for-2025-from-marcus-6d352a-a2: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-25-ai-predictions-for-2025-from-marcus-6d352a-a2.