“I said the problem wasn’t math per se, but that large language models didn’t represent robust world models of how events fold over time; I stand by this 100% even today.”
Our reading, the sentence that was checked:
Large language models do not represent robust world models of how events unfold over time, and this was true from 2019 through early 2023.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Gaslighting and reality in AI, paragraph 13.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-gaslighting-and-reality-in-ai-c17009-a1
We recorded 4 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-02-02. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- large language models
- didn’t represent robust world models
- how events fold over time
- I stand by this 100% even today
Our interpretation
- The context compares GPT-2 in October 2019 with language models available by February 2, 2023.
- The plain reading is that these models lacked dependable representations for tracking events and their consequences across varied descriptions. It does not require a complete absence of internal state information.
- Repeated behavioral failures bear on robustness, but do not by themselves establish what internal representations are absent.
Counts as supported if: Evaluations of representative models across the stated period show persistent failures to track novel event sequences and resulting states across ordinary variations, with evidence linking those failures to unreliable state representations.
Counts as contradicted if: Representative models available within the period demonstrably maintain and use dependable event-state representations across varied novel sequences and formulations. Success on a narrow task alone is insufficient.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The evidence supports a real reliability concern, but does not settle the full claim across models and years. Contemporary research found internal representations of changing situations, while also finding those representations imperfect. The Othello result supplies a further counterpoint, but concerns a specialized model. Neither narrow successes nor observed errors alone settle whether general-purpose LLMs lacked robust temporal world models throughout the stated period.
Limitations
- The conclusion depends partly on what counts as robust.
- The strongest direct representation evidence concerns restricted tasks and a limited set of models.
Evidence (2 items)
Supports the claim: arXiv; Belinda Z. Li, Maxwell Nye, and Jacob Andreas,
these implicit models are approximate and error-prone.
Why it matters here: The BART/T5 experiments support the narrower concern about unreliable state representations. They also find usable dynamic representations, so they do not support a literal absence of world models.
- Publisher
- arXiv; Belinda Z. Li, Maxwell Nye, and Jacob Andreas
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv; Kenneth Li and colleagues,
we find evidence for an emergent world representation.
Why it matters here: The study finds board-state representations in a GPT variant trained on Othello sequences and tests their causal role. This challenges categorical claims about sequence models, but does not demonstrate robust everyday event understanding in general-purpose LLMs.
- Publisher
- arXiv; Kenneth Li and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Matched evaluations of archived GPT-2 and early-2023 models across varied event sequences, combined with interventions testing whether their internal state representations reliably determine predictions.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Gaslighting and reality in AI
I said the problem wasn’t math per se, but that large language models didn’t represent robust world models of how events fold over time; I stand by this 100% even today.
- Where
- paragraph 13
- Published
- 2023-02-02
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-gaslighting-and-reality-in-ai-c17009-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-gaslighting-and-reality-in-ai-c17009-a1.