Ezra Klein’s new take on AGI – and why I think it’s probably wrong
We recorded 29 claims from this post. 16 were checked against evidence. 13 can’t be checked. 12 more are held. Read the post at its source.
Claims from this post (29)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Conditional forecast
“I completely agree with Klein that AGI will matter a lot when it comes.”
Our reading: When AGI arrives, it will matter a lot.
No horizon
Only if: when AGI arrives
Assessment: UnresolvedSaid
Causal claim
“But it is not just that I think Klein is wrong, or that he did a disservice to the other side of the argument, but that words like his are motivating a lot of dodgy bets being placed on the imminent AGI assumption — in ways that may not make sense.”
Our reading: Claims like Klein's that AGI is imminent are motivating many dodgy bets that may not make sense.
No horizon Evidence as of
Limitation: Public explanations may omit internal motives.
Assessment: SupportedSaid
Observation
“But it is absolutely absurd to pretend “AGI imminent” hypothesis isn’t getting play in high places.”
Our reading: The hypothesis that AGI is imminent is getting play in high places, and it is absurd to pretend otherwise.
No horizon Evidence as of
Limitation: This supports the general observation, not every example or institutional explanation in the surrounding post.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“And, importantly, the reverse of what Mollick said is also true: we have to consider the scenario that the proposition that AGI is imminent is false.”
Our reading: We have to consider the scenario in which the proposition that AGI is imminent is false.
No horizon
Assessment: UnresolvedSaid
Causal claim
“(My view, as I explained in Taming Silicon Valley, is that industry has a vested interest in making people believe that AGI is imminent — how else to justify the lavish valuations - and that they have fooled a lot of people in government who aren’t technical experts into believing that AGI is closer than it really is).”
Our reading: The AI industry has a vested interest in claiming AGI is imminent to justify lavish valuations, and has misled many non-technical government officials into thinking AGI is closer than it is.
No horizon Evidence as of
Limitation: One company's announcement cannot establish an industry-wide valuation mechanism.
Assessment: SupportedSaid
Observation
“Every red cell in this graph (e.g, multiplying 20 digit numbers by 19 digit numbers2) is failure to generalize, even after immense amounts of training data:”
Our reading: LLMs including o1-mini fail to generalize on large multi-digit multiplication despite immense training data.
No horizon Evidence as of
Limitation: The retrieved text does not permit a cell-by-cell audit, including the specific 20-by-19-digit example.
Assessment: SupportedSaid
Observation
“• AGI should have reliable ability to reason over abstractions. (Ongoing unpredictable errors in LLMs reflect this.)”
Our reading: LLMs still lack a reliable ability to reason over abstractions, as their ongoing unpredictable errors show.
No horizon Evidence as of
Limitation: The February studies are preprints and do not provide complete execution timestamps.
Assessment: SupportedSaid
Observation
“Hallucinations simply haven’t gone away, even in the latest models.”
Our reading: Hallucinations have not gone away, even in the latest models.
No horizon Evidence as of
Limitation: PersonQA concerns factual questions about people; its rate is not an estimate for all uses.
Assessment: UnresolvedSaid
Observation
“• AGI should have a reliable ability to understand complex syntactic and semantic structure. (Failings here are also often reflected in quirky errors.”
Our reading: Current AI systems lack a reliable ability to understand complex syntactic and semantic structure, and these failings are often reflected in quirky errors.
No horizon Evidence as of
Limitation: This is an evidence gap, not a finding that the alleged errors do not occur.
Assessment: UnresolvedSaid
Observation
“What that means is that a quarter century has passed without a principled solution to even one of those problems.”
Our reading: In the quarter century since 2001, none of the listed AGI prerequisites (generalization, abstract reasoning, tracking individuals, understanding structure) has received a principled solution.
No horizon Evidence as of
Limitation: A narrow successful algorithm would not refute the contextual claim about general capabilities.
Assessment: UnresolvedSaid
Observation
“Especially when we just saw GPT-5 (ultimately relabeled as GPT 4.5) fail despite more than two years of hard and expensive work.”
Our reading: OpenAI's planned GPT-5, relabeled GPT-4.5, failed despite more than two years of hard and expensive work.
No horizon Evidence as of
Limitation: Product improvements can coexist with missed internal expectations.
Assessment: Can’t be checkedSaid
Forecast
“A set of problems that have resisted solution for a quarter century, even backed by a half trillion dollars, probably aren’t all going to get solved so quickly.”
Our reading: The long-standing core AGI problems probably will not all be solved quickly, even with half a trillion dollars of funding.
No horizon
Qualifier: even backed by a half trillion dollars
Assessment: Can’t be checkedSaid
Forecast
“Having observed the pace of science and engineering firsthand over the last half century, the middle scenario seems to me like – by far – the most plausible.”
Our reading: The most plausible scenario by far is that getting to AGI will be a slog: some problems get solved in the next 2-3 years and others take longer.
No horizon
Assessment: UnresolvedSaid
Forecast
“But there’s also no reason to think that any of them are logically insolvable in principle. So we will get there someday.”
Our reading: Because there is no reason to think any of the remaining problems are logically unsolvable in principle, AGI will be achieved someday.
No deadline stated Evidence as of
Limitation: The search cannot establish that every possible demonstration has been examined.
Assessment: UnresolvedSaid
Observation
“The fact that GPT 4.5 received very mixed reviews — despite years of labor and billions investment — and could in no reasonable way be considered a major (rather than minor, incremental) step forwards towards AGI.”
Our reading: GPT-4.5 received very mixed reviews and is at most a minor, incremental step toward AGI, despite years of work and billions invested.
No horizon Evidence as of
Limitation: Company benchmarks and limited pre-release evaluations do not exhaust a model's capabilities.
Assessment: Can’t be checkedSaid
Conditional forecast
“The fact that GPT 4.5 is wildly expensive relative to its competitors, and the fact that future models following the same trajectory of scale might be prohibitively expensive, especially in a regime in which nobody has a moat and price wars have been intensifying since the release of DeepSeek.”
Our reading: Future models that keep scaling the same way might be prohibitively expensive, especially since nobody has a moat and price wars are intensifying.
No horizon
Only if: if future models follow the same trajectory of scale
Assessment: UnresolvedSaid
Observation
“The fact that Grok 3 was 15x the size of Grok 2, and also generally viewed as disappointing.”
Our reading: Grok 3 was 15 times the size of Grok 2 and was generally viewed as disappointing.
No horizon Evidence as of
Limitation: The scale judgment depends on what size means.
Assessment: Can’t be checkedSaid
Observation
“Automatically generated bullshit still seems to be the norm.”
Our reading: Automatically generated bullshit still seems to be the norm, including in systems like Deep Research and Grok 3 Deep Search.
No horizon
Qualifier: seems
Assessment: UnresolvedSaid
Forecast
“And these complex systems, built on the unreliable foundations of LLMs, are going to prove to be massively difficult to debug.”
Our reading: Complex systems built on LLMs, such as deep research agents, will prove massively difficult to debug.
No deadline stated Evidence as of
Limitation: The production account is a company report without quantified debugging effort.
Assessment: UnresolvedSaid
Forecast
“AI might replace average people in some tasks soon; replacing the best humans in a few years? Not going to happen.”
Our reading: AI will not replace the best humans, such as top scientists, within a few years, though it might replace average people in some tasks soon.
Not yet due Evidence as of
Limitation: The exact deadline and meaning of replacement are interpretive.
Assessment: SupportedSaid
Observation
“Hallucinations haven’t solved, despite constant assurances.”
Our reading: Hallucinations have not been solved despite constant assurances that they would be.
No horizon Evidence as of
Limitation: PersonQA is a company-reported, task-specific evaluation; its rates do not establish rates across everyday use.
Assessment: SupportedSaid
Observation
“Most parts of the US still don’t have functioning driverless cars systems, for example, even after over a decade of lavish promises.”
Our reading: Most parts of the US still lack functioning driverless car systems after more than a decade of lavish promises.
No horizon Evidence as of
Limitation: Geographic coverage is assessed qualitatively rather than through a national area or population calculation.
Assessment: Can’t be checkedSaid
Conditional forecast
“If LLMs are the wrong path to AGI, as I and a great many academics think, we may have a lot of thinking to do. We may be massively wasting resources, and getting locked into a lot foolish battles around the wrong technology.”
Our reading: If LLMs are the wrong path to AGI, as Marcus and a great many academics think, we may have a lot of thinking to do and may be massively wasting resources and getting locked into foolish battles around the wrong technology.
No horizon
Only if: if LLMs are the wrong path to AGI
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“In a time when Trump is dismantling American research infrastructure, it should have been front and center.”
Our reading: Research should have been front and center in the Klein–Buchanan episode, at a time when Trump is dismantling American research infrastructure.
No horizon
Qualifier: in a time when Trump is dismantling American research infrastructure
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If getting to AGI in the next three years is not quite as certain as the sun rising every day from the east, we might want to hedge our bets, and try to focus on what the field of AI need to develop besides scaling.”
Our reading: If getting to AGI in the next three years is not quite certain, we might want to hedge our bets and try to focus on what the field of AI needs to develop besides scaling.
No horizon
Only if: if getting to AGI in the next three years is not quite as certain as the sun rising
Assessment: Can’t be checkedSaid
Forecast
“I couldn’t help but think that the US’s single-minded commitment to Large Language Models, predicated on implausible assumptions about AGI’s imminence, particularly combined with the untimely gutting of its scientific institutions, could ultimately lead to its undoing.”
Our reading: The US's single-minded commitment to LLMs, based on implausible assumptions about AGI's imminence and particularly when combined with the untimely gutting of its scientific institutions, could ultimately lead to the US's undoing.
No horizon
Qualifier: particularly combined with the untimely gutting of its scientific institutions
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If it is this unreliable on arithmetic, it will be this unreliable on everything else, and we shouldn’t accord it the society-changing power that people intend to accord it.”
Our reading: An AI system this unreliable on arithmetic will be similarly unreliable on everything else and should not be given society-changing power.
No horizon
Qualifier: if it is this unreliable on arithmetic
Assessment: Can’t be checkedSaid
Forecast
“I am not convinced that anything anyone could realistically do with LLMs won’t be almost immediately copied, as has been the recent trend.”
Our reading: Marcus is not convinced that anything anyone could realistically do with LLMs won't be copied almost immediately, as has been the recent trend.
No horizon
Qualifier: not convinced
Assessment: Can’t be checkedSaid
Observation
“It actually seems to me that US may be all but surrendering the AGI race over to China.”
Our reading: With its academic research institutions being dismantled, the US may be all but surrendering the AGI race to China.
No horizon
Qualifier: given the dismantling of academic research institutions
Held from this post (12)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- Attribution is uncertain
- The criteria are ambiguous. Our reading, not assessed: There is almost zero chance that AGI, meaning a system that can do almost any cognitive task a human can do, will arrive within the next two to three years.
- The record does not meet the current rules. Our reading, not assessed: The hypothesis that AGI is imminent is almost certainly wrong.
- The criteria are ambiguous. Our reading, not assessed: OpenAI's Orion appears to have been downgraded and released as GPT-4.5 rather than GPT-5.
- The two reviews reached different assessments. Our reading, not assessed: AGI timelines are the focus of some of the largest investments in history and central to many delicate policy questions.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: The policies Klein and Buchanan discuss throughout the episode, such as chip export controls and massive infrastructure investments, rest almost entirely on the assumption that AGI is nigh and that LLMs will get us there.
- The two reviews reached different assessments. Our reading, not assessed: Apple's recent study on formal reasoning very much confirms that current AI systems still lack a reliable ability to freely generalize abstractions to new instances, more than two decades after Marcus first pointed out this weakness.
- Attribution is uncertain
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: The scenario in which AGI is not imminent is underrepresented in current policy discussions.