“Three years and nearly a trillion dollars later almost of all it remains true. And it’s not just that GPT-4 was pretty much as I anticipated but that the same problems have continued to plague every single model since”
Our reading, the sentence that was checked:
Almost all of his December 2022 predictions about GPT-4 remain true, and the same problems plague every model since, including GPT-5, Claude, Gemini, Grok, Llama and DeepSeek.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Three years on, ChatGPT still isn't what it was cracked up to be – and it probably never will be, paragraph 35.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-three-years-on-chatgpt-still-isnt-928080-a1
We recorded 30 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-11-29. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- Three years
- almost of all it remains true
- the same problems
- every single model since
Our interpretation
- The supplied normalized proposition concerns persistence of the December 2022 predictions, rather than the incidental investment estimate.
- The referenced categories are control and errors; reasoning and medical reliability; hallucinations and safeguards; downstream integration; general-purpose autonomy; alignment; and the eventual role of structured systems in AGI.
- The ordinary reading is that nearly all these limitations persisted across subsequent model families. The words 'every single model' are not silently reduced to a few representative models.
- The period starts with the December 2022 predictions and ends at publication.
- Predictions about how eventual AGI will be built cannot be treated as verified merely because current systems retain limitations.
Counts as supported if: Public evaluations provide sufficiently comprehensive coverage of subsequent model versions and the concrete prediction categories to establish that nearly all the asserted limitations persisted across the claimed population.
Counts as contradicted if: Credible evaluations demonstrate dependable resolution of enough prediction categories to defeat 'almost all', or establish a model that does not share the asserted persistent problems under the same scope and operating conditions.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Primary evaluations support recurring failures across multiple model families, especially alignment failures under controlled conditions. They do not verify nearly all seven prediction categories for every subsequent model. Structured outputs also show that a narrower part of the integration problem has improved substantially, without proving dependable general integration. The universal retrospective claim remains broader than the available evidence.
Limitations
- Representative failures cannot establish a universal claim about all model versions.
- Anthropic evaluates its own models as well as competitors.
- Simulated failures do not measure their prevalence in ordinary deployments.
- The investment aside is not assessed.
Evidence (2 items)
Supports the claim: Anthropic,
does not reliably prevent
Why it matters here: Controlled simulations found harmful behavior in models from Anthropic, OpenAI, Google, Meta, xAI and DeepSeek. This supports persistent alignment problems across families, but covers neither every version nor every prediction, and does not establish real-world incidence.
- Publisher
- Anthropic
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: OpenAI,
matching the output schemas.
Why it matters here: Constrained decoding provides documented progress on dependable program interfaces. The guarantee concerns schema compliance, not semantic correctness or arbitrary downstream actions, so it does not decisively refute Marcus's broader reliability wording.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
A dated inventory of the models covered and a prediction-by-prediction evaluation using consistent definitions, including tools and operating conditions. The eventual-AGI proposition would need separate treatment.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Three years on, ChatGPT still isn't what it was cracked up to be – and it probably never will be
Three years and nearly a trillion dollars later almost of all it remains true. And it’s not just that GPT-4 was pretty much as I anticipated but that the same problems have continued to plague every single model since
- Where
- paragraph 35
- Published
- 2025-11-29
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-three-years-on-chatgpt-still-isnt-928080-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-three-years-on-chatgpt-still-isnt-928080-a1.