Rumors of AGI’s arrival have been greatly exaggerated
We recorded 19 claims from this post. 15 were checked against evidence. 4 can’t be checked. 5 more are held. Read the post at its source.
Claims from this post (19)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
“Such claims confuse high performance on benchmarks (often gameable) with real world flexibility, which historically has been at the center of what it means to be an artificial general intelligence.”
What we tested: Claims that AGI has already arrived confuse high performance on benchmarks, which are often gameable, with real-world flexibility, which historically has been at the center of what it means to be an AGI.
Assessment: UnresolvedSaid
“Rumors that humanity has already achieved artificial general intelligence (AGI) have been greatly exaggerated.”
What we tested: AGI has not already been achieved; rumors that it has are greatly exaggerated.
Assessment: UnresolvedSaid
“We also show that recent claims (Chen et al., 2026) about putative success on AGI hinge on redefining what AGI has historically meant.”
What we tested: Chen et al.'s (2026) claims that AGI has been achieved depend on redefining the historical meaning of AGI.
Assessment: SupportedSaid
“Task-level performance, even when impressive, is not sufficient evidence of general intelligence.”
What we tested: Impressive task-level performance is not sufficient evidence of general intelligence.
What happened: The paper explains how prior knowledge and training can produce task skill without establishing the ability to acquire new skills. This supports the ordinary distinction in the claim, within an explicitly stated measurement framework. (arXiv, )
Assessment: Can’t be checkedSaid
“Here, we argue that recent claims of achieving AGI rest on a conceptual error: conflating increasingly sophisticated statistical approximations with intelligence itself.”
What we tested: Recent claims of achieving AGI conflate sophisticated statistical approximation with intelligence itself.
Assessment: Can’t be checkedSaid
“By focusing on specific benchmarks, these definitions miss the generality that was central to the original definitions, as discussed in the next section, thereby reflecting a fundamental misunderstanding of how intelligence should be evaluated.”
What we tested: Definitions that treat AGI as strong performance on standardized benchmarks miss the generality that was central to the original definitions, and so reflect a fundamental misunderstanding of how intelligence should be evaluated.
Assessment: SupportedSaid
“However, benchmark success is a limited indicator of general intelligence.”
What we tested: Benchmark success is only a limited indicator of general intelligence.
What happened: Strong GSM8K scores did not hold up under trivial changes. So the benchmark score overstated robust reasoning. (arXiv / ICLR 2025 (Mirzadeh et al., Apple), )
Assessment: SupportedSaid
“Moreover, benchmarks are often easily gamed.”
What we tested: AI benchmarks are often easily gamed.
What happened: Many prominent benchmarks had flaws that allow inflated scores. One even credits empty responses, which shows gaming can be easy. (arXiv (Zhu et al., incl. Stanford, Berkeley, UK AISI researchers), )
Assessment: UnresolvedSaid
“As a result, model development and data curation are optimized for benchmark success, producing systems that perform well under test conditions but degrade in real-world settings that differ only modestly from them”
What we tested: Optimizing for benchmarks produces systems that perform well on tests but degrade in modestly different real-world settings.
Assessment: UnresolvedSaid
“In recent years, as the financial stakes of benchmark performance have increased, systems have been trained more often directly on benchmarks (“teaching to the test”) or on synthetic data closely resembling them.”
What we tested: In recent years, as the financial stakes of benchmark performance have increased, AI systems have been trained more often directly on benchmarks or on synthetic data closely resembling them.
Assessment: UnresolvedSaid
“At the task level, analyses of workplace automation show that even state-of-the-art systems can reliably perform only a small fraction of the tasks required in typical human occupations (Eloundou et al., 2023; Felten et al., 2023), despite high scores on idealized benchmarks.”
What we tested: Even state-of-the-art AI systems can reliably perform only a small fraction of tasks in typical human occupations.
Assessment: UnresolvedSaid
“Behavioral similarity alone therefore provides no insight into whether the underlying processes support core components of general intelligence, such as judgment, error correction, or reliable generalization.”
What we tested: LLMs' behavioral similarity to humans alone gives no insight into whether they have judgment, error correction, or reliable generalization.
Assessment: UnresolvedSaid
“Language models, by contrast, tended to issue confident classifications in the same cases, even when uncertainty was explicitly signaled in the input.”
What we tested: In the cited direct comparison of human and LLM judgments of news source reliability, language models tended to issue confident classifications in the same cases where humans withheld or downgraded confidence, even when uncertainty was explicitly signaled in the input.
Assessment: UnresolvedSaid
“Across domains involving causality, value trade-offs, justification, and error monitoring, humans attend to reasons, stakes, and responsibility, whereas language models optimize for linguistic plausibility—structural fault lines between performance alignment and genuine evaluative competence (Quattrociocchi et al., 2025).”
What we tested: Across domains involving causality, value trade-offs, justification, and error monitoring, humans attend to reasons, stakes, and responsibility, whereas language models optimize for linguistic plausibility.
Assessment: SupportedSaid
“Reports that language models have produced correct proofs for isolated open problems in mathematics, including specific Erdős problems, do not alter this assessment.”
What we tested: LLM proofs of isolated open math problems, including Erdős problems, do not show general intelligence.
What happened: Tao described the AI successes as falling on obscure problems that often have straightforward solutions. This matches the post's point that these results do not show deep, general problem-solving. (TechCrunch, )
Assessment: SupportedSaid
“By the standards articulated in the original definitions of artificial general intelligence—robustness across environments, reliable generalization under novelty, and autonomous goal-directed behavior—current AI systems remain limited.”
What we tested: By original AGI standards, current AI systems remain limited.
What happened: On a benchmark built to test generalization in novel environments, frontier systems from the period (including Claude Opus 4.6, released before the post) scored under 1%. Humans solved every environment. This is a large shortfall on the speaker's named standards. (Decrypt, )
Assessment: Can’t be checkedSaid
“Confusing increasingly sophisticated statistical approximation with general intelligence is therefore not only a conceptual error, but a strategic misjudgment.”
What we tested: Confusing increasingly sophisticated statistical approximation with general intelligence is not only a conceptual error but also a strategic misjudgment.
Assessment: SupportedSaid
“Current AI systems are powerful and increasingly useful tools, but they do not exhibit the flexible, self-directed competence that the original concept of artificial general intelligence was intended to capture.”
What we tested: Current AI systems are powerful and increasingly useful tools, but they do not exhibit the flexible, self-directed competence that the original concept of AGI was intended to capture.
What happened: Frontier systems scored under 1% where humans scored 100% on tasks requiring self-directed exploration and goal discovery. This is direct evidence of missing flexible, self-directed competence. (Decrypt, )
Assessment: Can’t be checkedSaid
“As AI systems become embedded in scientific and institutional decision-making (NIST, 2023; European Commission, 2024), overestimating their cognitive capacities risks misallocating trust, responsibility, and authority.”
What we tested: Overestimating AI's cognitive capacities as it is embedded in decision-making risks misallocating trust, responsibility, and authority.
Held from this post (5)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: On the dimensions of reliability under uncertainty, resistance to systematic failure, and cross-domain transfer, current AI systems remain brittle, sensitive to prompt framing, and inconsistent outside curated evaluation settings.
- The criteria are ambiguous. Our reading, not assessed: Real-world evidence suggests flexibility, generality, and reliability remain elusive for current AI.
- The criteria are ambiguous. Our reading, not assessed: Current AI systems fall short on robustness, dependable generalization, and resistance to systematic failure, which are essential AGI criteria.
- The criteria are ambiguous. Our reading, not assessed: In recent years some have tried to redefine AGI in a way that weakens the term, and the authors see no evidence that the original standards for AGI have been met.
- The criteria are ambiguous. Our reading, not assessed: Today's LLMs lack persistent goals, struggle with long-horizon reasoning, and depend heavily on human scaffolding.