“It turns out that attention, the key component in LLMs, and the focus of the justly famous Transformer paper, is not fact “all you need”.”
Our reading:
Attention, the key component in LLMs, is not in fact 'all you need' to get to AGI.
Assessment: Unresolved
What happened
Did the forecast come true?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
ARC Prize's September 3, 2026 report gave GPT-6 Astra 62.7% on ARC-AGI-3 with its Standard harness and 99.9% with a Provider Adapter harness [e2]. These results concern unfamiliar interactive environments and do not establish broadly human-level general intelligence across domains [e2]. The AGI Society's September 2026 review judged the published evidence insufficient to conclude that Astra had achieved AGI [e3]. The original Transformer paper evaluated translation and parsing, without establishing AGI or its impossibility [e1]. None of these items supplies the required architectural AGI demonstration or a general impossibility result, leaving the forecast unresolved [e1, e2, e3].
Caveats
- ARC-AGI-3 covers a bounded set of tasks, and the reported results differ by harness.
- The AGI Society review assesses published evidence. It is not a direct AGI test.
- The packet does not establish whether separately added representational mechanisms are necessary.
- Model-generated symbolic representations do not by themselves establish separately added architectural components.
- Current failures or missing evidence cannot establish an indefinite architectural limit.
Evidence (3 items)
Context only: Vaswani et al., arXiv,
through an attention mechanism.
Why it matters: The paper presents an attention-based sequence architecture and evaluates translation and parsing. Its title does not establish that attention suffices for AGI, so Marcus's allusion cannot be settled merely by checking the original paper.
- Publisher
- Vaswani et al., arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Context only: ARC Prize Foundation,
on 96% of levels.
Why it matters: This is material evidence of progress on unfamiliar interactive tasks. However, ARC Prize expressly declines to identify Astra as AGI, describes the benchmark's bounded scope, and reports generated symbolic models and different harness conditions; it does not establish the required architectural counterexample.
- Publisher
- ARC Prize Foundation
- Kind of source
- Independent measurement
- About events on
- Published
Context only: AGI Society,
Our current evaluation
Why it matters: This independent assessment considers the later AGI declarations insufficiently established by published evaluations. It is a review of evidence, not a direct AGI test, and does not prove Marcus's indefinite architectural prediction.
- Publisher
- AGI Society
- Kind of source
- Secondary summary
- About events on
- Published
How we judged it
- Kind of claim
- Forecast
- Deadline
- None given. We don’t add one.
Where we had to interpret
- Read in context, this predicts that scaling attention-based LLMs without additional representational mechanisms will not achieve AGI.
- The ordinary reading concerns attention-based LLM architectures, including their usual supporting components, rather than an isolated attention operation.
- I interpret AGI as broadly human-level ability to learn and perform unfamiliar cognitive tasks across domains. Marcus supplies no numerical threshold or agreed test.
- A qualifying counterexample needs independent capability evaluation and sufficient architectural disclosure to establish that separately added symbolic or world-model components were unnecessary.
- Representations learned or generated by the model do not automatically count as separately added architectural components.
- The claim has no deadline. Continued failures or success by a different architecture cannot establish that this approach will never suffice.
Right if: An independently scrutinized general result establishes that attention-based LLMs cannot attain broadly human-level general intelligence without additional representational mechanisms, under assumptions that cover the claimed approach. Failures of particular models alone are insufficient.
Wrong if: A publicly documented attention-based LLM achieves independently demonstrated, broadly accepted human-level general intelligence without separately added symbolic or explicit world-model components.
We wrote this test after the outcome was known.
What would settle it
An independently verified, broadly accepted demonstration of human-level general intelligence by an attention-based LLM, with disclosure establishing that separately added symbolic or explicit world-model components were unnecessary; or an independently scrutinized general impossibility result covering that approach.
We found 33 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it., paragraph 87.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-08-09.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
It turns out that attention, the key component in LLMs, and the focus of the justly famous Transformer paper, is not fact “all you need”.
- Where
- paragraph 87
- Published
- 2025-08-09
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
The final check failed: a cited source didn’t say what the write-up said.