“So far as I know even the latest language models are still bulls in a china shop, powerful but hard to control; they still can’t reason reliably; they still don’t work reliably with external tools; they continue to hallucinate; they still can’t match domain specific models, the continue to struggle with alignment.”
Our reading:
So far as Marcus knows, even the latest language models are still powerful but hard to control, still can't reason reliably, still don't work reliably with external tools, continue to hallucinate, still can't match domain-specific models, and continue to struggle with alignment.
Assessment: Can’t be checked
What happened
Not graded on evidence
Outcome: Can’t be checked
An opinion, a recommendation, or too vague to test. Listed, not scored.
Evidence up to . Checked .
Why
The statement qualifies the whole list with “so far as I know” and gives no reliability thresholds or named specialist comparators. These qualifications leave the complete statement without a defined public-evidence test [marcus-three-years-on-chatgpt-still-isnt-9711be-o1].
Caveats
- The packet contains no evidence items testing the listed technical limitations.
- This assessment establishes neither that the individual limitations hold nor that they are false.
- Public model evaluations cannot establish the boundaries of the speaker’s knowledge.
Evidence (0 items)
Claims like this aren’t scored, so there is no evidence to show.
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- so far as I know (their words)
- even the latest language models (their words)
Where we had to interpret
- Preserve the knowledge qualification governing the whole list. Do not turn this into an unqualified assertion about every frontier model.
- The statement describes the latest models at publication. November 1–29, 2025 is the current observation window; models have no standard quarterly reporting period.
- Public evaluations can test individual limitations, but cannot establish the boundaries of Marcus’s knowledge. He also supplies no common reliability threshold or specified domain-specific comparison.
- A binary assessment of this entire qualified list would require commitments beyond his words. The firm statement about persistent problems is assessed separately.
We don’t grade claims like this. We only check the quote and who said it.
We found 34 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Three years on, ChatGPT still isn't what it was cracked up to be – and it probably never will be, paragraph 38.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-11-29.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Three years on, ChatGPT still isn't what it was cracked up to be – and it probably never will be
So far as I know even the latest language models are still bulls in a china shop, powerful but hard to control; they still can’t reason reliably; they still don’t work reliably with external tools; they continue to hallucinate; they still can’t match domain specific models, the continue to struggle with alignment.
- Where
- paragraph 38
- Published
- 2025-11-29
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Can’t be checked. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.