“More recently, Hendrycks et al. (2025) reaffirmed the importance of robustness, dependable generalization beyond curated benchmarks, and resistance to systematic failure as essential criteria for AGI—criteria on which current systems fall short.”
Our reading:
Current AI systems fall short on robustness, dependable generalization, and resistance to systematic failure, which are essential AGI criteria.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The cited framework requires more than success on illustrative tests, and the contemporary safety report documents continuing failures. That makes Marcus's assessment plausible. But these records do not separately establish the three asserted deficits across the frontier available when he wrote. A newer model also claimed relevant improvements. An aggregate AGI score or evaluations of older models would not close that gap.
Caveats
- The available synthesis is broader than the required property-by-property evaluation.
- Company-reported progress is not independent confirmation that the requirements were met.
- This finding does not establish either that AGI existed or that it was absent.
Evidence (3 items)
Context only: arXiv,
examples are not exhaustive.
Why it matters: The framework distinguishes solving examples from possessing the underlying ability and separately recommends testing distribution shifts. This establishes relevant criteria, not an independent finding that contemporary systems fail them.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Context only: International AI Safety Report,
Performance nevertheless remains ‘jagged’
Why it matters: The report corroborates persistent weaknesses and discusses failures on unusual tasks and gaps between evaluations and deployment. It does not separately establish all three asserted deficits for systems introduced after its publication.
- Publisher
- International AI Safety Report
- Kind of source
- Secondary summary
- About events on
- Published
Context only: Anthropic,
retrieving relevant information
Why it matters: Anthropic reports improved long-context performance for a model released before the post. This is material evidence of progress and shows why older evaluations cannot automatically settle the claim, but it does not demonstrate all three AGI requirements.
- Publisher
- Anthropic
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- The ordinary reading is that contemporary general-purpose AI still has material weaknesses in all three named properties. It does not require error-free performance.
- Robustness means retaining competence under modest changes in inputs or conditions. Generalization means transferring competence beyond familiar test tasks. Resistance to systematic failure means avoiding recurring classes of failure.
- Current systems includes the frontier available at publication, not just models evaluated in the cited 2025 paper.
- With no reporting quarter for AI capabilities, the target is the publication month through the post date: February 1–17, 2026. Earlier evaluations provide context unless they establish capabilities during that interval.
- The paper's aggregate cognitive score is not a substitute for measurements of these three properties.
Right if: Independent evaluations covering contemporary leading systems show material, recurring weaknesses in robustness, transfer beyond curated tests, and resistance to systematic failure. The cited paper also treats these properties as requirements rather than optional improvements.
Wrong if: Contemporary systems demonstrably satisfy the three requirements across varied unfamiliar tasks and realistic conditions, or the cited paper explicitly rejects their necessity. Strong scores on unrelated benchmarks alone do not suffice.
We wrote this test after the outcome was known.
What would settle it
Dated evaluations of the publication-date frontier that test modest perturbations, unfamiliar settings, and recurring failure patterns, with enough breadth to assess each named requirement.
We found 22 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Rumors of AGI’s arrival have been greatly exaggerated, paragraph 13.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2026-02-17.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Rumors of AGI’s arrival have been greatly exaggerated
More recently, Hendrycks et al. (2025) reaffirmed the importance of robustness, dependable generalization beyond curated benchmarks, and resistance to systematic failure as essential criteria for AGI—criteria on which current systems fall short.
- Where
- paragraph 13
- Published
- 2026-02-17
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.