“But we see no reason to think that current large language models encode stable representations of the world at all. All they are doing, literally, is predicting next words in sentences.”
Our reading:
The authors see no reason to think that current large language models encode stable representations of the world at all, and say that all these models are literally doing is predicting the next words in sentences.
Assessment: Can’t be checked
What happened
Not graded on evidence
Outcome: Can’t be checked
An opinion, a recommendation, or too vague to test. Listed, not scored.
Evidence up to . Checked .
Why
The qualification is material. This is an assessment of whether there is sufficient reason to attribute stable world representations, not an unqualified claim that no internal representations exist. Scoring a probe result as a contradiction would remove that qualification and supply a definition of stability. The next-word statement can be examined separately, but it does not make the combined assessment uniquely testable.
Caveats
- This does not endorse the inference from next-word prediction to an absence of world representations.
- A narrower claim about a specified model and a defined form of stability could be assessed.
Evidence (0 items)
Claims like this aren’t scored, so there is no evidence to show.
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- we see no reason to think (their words)
- current large language models (their words)
Where we had to interpret
- Preserve the authors' qualified assessment of the evidence; do not replace it with a categorical denial that any internal world-related information exists.
- The next-word statement is presented as the rationale for that assessment. A training objective alone does not specify what intermediate representations a model can learn.
- The passage supplies neither a criterion for stable world representations nor a standard for evidence sufficient to change the authors' assessment.
- A verdict on the combined claim would therefore require adding a consequential standard the speaker did not supply.
We don’t grade claims like this. We only check the quote and who said it.
What would settle it
Clarification of the intended stability criterion and whether the authors mean an objective absence of qualifying evidence or their own assessment of its persuasiveness.
We found 33 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Three ideas from linguistics that everyone in AI should know, paragraph 15.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-29. The post may have been edited after 2022-06-22.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Three ideas from linguistics that everyone in AI should know
But we see no reason to think that current large language models encode stable representations of the world at all. All they are doing, literally, is predicting next words in sentences.
- Where
- paragraph 15
- Published
- 2022-06-22
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Can’t be checked. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.