“Everything else is glommed on top, with more or less reliabilty.”
Our reading:
Capabilities beyond word correlation are bolted onto LLMs with variable reliability.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The records document corrective training and an update that produced unwanted behavior. These fit parts of Marcus's description. Research also shows that sequence prediction can produce useful internal representations, although the cited experiment concerns a board game. None of this settles the broad claim about where practical capabilities originate. Including training-time corrections avoids treating capabilities learned inside model weights as an automatic refutation.
Caveats
- The company disclosures concern earlier systems and are not independent audits.
- The controlled representation study has a narrow synthetic setting.
- The claim supplies no precise boundary between underlying capabilities and additions.
Evidence (3 items)
Context only: OpenAI,
shapes the behavior
Why it matters: OpenAI describes post-training that changes model behavior, including human and model-generated examples. This fits the inclusive reading of additional engineering but does not establish that all practical capabilities originate there.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
Context only: OpenAI,
overly flattering or agreeable
Why it matters: This is a documented instance of a behavioral update producing an unwanted result. It supports concern about the reliability of interventions, but concerns an earlier deployment and cannot establish the entire observation.
- Where
- Opening paragraph
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
Context only: Li and colleagues; ICLR 2023, revised manuscript on arXiv,
used to control the output of the network.
Why it matters: The Othello experiments found an internal board representation learned through sequence prediction, which cautions against treating initial training as mere surface correlation. A synthetic board game does not establish reliable real-world updating in deployed assistants.
- Where
- Abstract
- Publisher
- Li and colleagues; ICLR 2023, revised manuscript on arXiv
- Kind of source
- Peer-reviewed paper
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- The plain reading is that practical capabilities beyond the underlying language-prediction process depend on additional engineering whose reliability varies.
- Additional engineering includes training-time corrections, fine-tuning, reinforcement learning, prompts, retrieval, tools, and other system components. Learning inside model weights does not by itself contradict the claim.
- The claim describes systems at publication, not a specified calendar year. The target is their state on 2026-01-03. Earlier experiments and disclosures provide context; they are not automatically evidence of that day's system state.
- A finding that systems use additional engineering establishes only part of the claim. Assessing 'everything else' also requires distinguishing capabilities acquired during initial training from capabilities supplied by subsequent interventions.
Right if: Architectural records and controlled evaluations of representative LLM systems show that their practical capabilities beyond language prediction depend on additional training or system components, with reliability varying across tasks or conditions.
Wrong if: Controlled evaluations establish that capabilities central to the claim arise reliably from initial language-model training without the additional corrective training or system components described here. Merely locating a capability inside model weights is insufficient.
We wrote this test after the outcome was known.
What would settle it
Controlled comparisons of representative systems available at publication, tracing capabilities through initial training, corrective training, and added components, with reliability measured on unfamiliar tasks.
We found 9 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Why ChatGPT can’t be trusted with breaking news, paragraph 12.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2026-01-03.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Why ChatGPT can’t be trusted with breaking news
Everything else is glommed on top, with more or less reliabilty.
- Where
- paragraph 12
- Published
- 2026-01-03
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.