“Large language models don’t produce such cognitive models, at least not in a way that anybody has been able to make reliable use of.”
Our reading:
Large language models do not produce cognitive models of the world, at least none that anyone has been able to use reliably.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
Prepublication research found updating, editable representations in natural-language models and accurate state tracking in specialized chess models. Those results matter, but they do not clearly establish the reliable use qualified in this claim. Conversely, limited reliability in particular experiments cannot establish that nobody had achieved such use. The evidence does not settle the qualified observation.
Caveats
- No arbitrary calendar-year restriction was imposed.
- Neither detecting a representation nor documenting failures in a few models settles the whole claim.
- The result depends partly on what counts as reliable use of a cognitive model.
Evidence (2 items)
Context only: arXiv,
These internal representations are structured, interpretably localized, and editable.
Why it matters: Li, Nye, and Andreas demonstrate state representations and interventions in BART and T5, which challenges a simple absence claim. Their limitations section reports complete-state recovery of only 53.8% even in the best case and leaves richer situations untested, so reliable functional use remains unsettled.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Context only: arXiv,
Unlike natural language, chess notations describe a simple, constrained, and deterministic domain.
Why it matters: The authors report accurate piece tracking and legal-move prediction from move sequences. This is relevant evidence of functional state tracking, but the specialized chess setting does not establish reliable cognitive-model use by the natural-language LLMs discussed in the post.
- Where
- Abstract
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Probably right?
One AI thinks this is probably right, and the other says it’s too early to tell. The evidence isn’t conclusive, so the verdict above stays unresolved. It counts only when you pick Speculative under “How sure?”.
- Anthropic’s AI: The best pre-2022 candidate, Li, Nye & Andreas (2021), recovered full states only 53.8% of the time, and its authors leave the link to model behaviour open. The chess study succeeds, but on chess notation, not natural language.
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- at least not in a reliably usable way (their words)
Where we had to interpret
- Assess whether reliable use had been demonstrated by publication, including earlier demonstrations without an arbitrary starting year.
- A cognitive model here maintains and updates representations of entities, their properties, and their relationships as a discourse unfolds.
- Reliable use means repeatable use of those representations for understanding, inference, or action. Decoding information from hidden states alone is insufficient.
- A bounded research demonstration can qualify; commercial deployment, human equivalence, and memory across separate sessions are not required.
- The target period is the accumulated demonstration record through 2022-06-22. Research capabilities have no quarterly reporting period.
Right if: A sufficiently comprehensive contemporary assessment establishes that available demonstrations had not achieved reliable functional use of LLM-produced, dynamically updated world representations.
Wrong if: At least one demonstration available by publication reliably uses a natural-language LLM's own updating representations of entities and relationships for understanding, inference, or action.
We wrote this test after the outcome was known.
What would settle it
A prepublication demonstration with reproducible downstream use and reliability measurements for an LLM-produced updating world model, or a comprehensive contemporary assessment establishing the absence of such demonstrations.
We found 33 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Three ideas from linguistics that everyone in AI should know, paragraph 6.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-29. The post may have been edited after 2022-06-22.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Three ideas from linguistics that everyone in AI should know
Large language models don’t produce such cognitive models, at least not in a way that anybody has been able to make reliable use of.
- Where
- paragraph 6
- Published
- 2022-06-22
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.