“But in other ways, large language models are a profound step backwards; they lack stable models of the world, which means that they are poor at planning, and unable to reason reliably; everything becomes hit or miss..”
Our reading, the sentence that was checked:
LLMs lack stable world models, which makes them poor at planning and unable to reason reliably.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Hello, Multimodal Hallucinations, paragraph 24.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-hello-multimodal-hallucinations-c3ae37-a1
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-10-21. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- lack stable models of the world
- which means that
- poor at planning
- unable to reason reliably
- everything becomes hit or miss
Our interpretation
- Assess the causal proposition about LLMs available when the post appeared, rather than treating 'a profound step backwards' as a measurable ranking.
- A stable world model means a representation that consistently tracks relevant entities, relations, and changes across tasks and equivalent descriptions.
- Reliable reasoning means dependable performance across relevant variations, not perfection on every possible problem.
- Use 2023 through the post date for contemporary evaluations. Later model improvements cannot settle this historical causal claim.
- Planning failures alone establish a behavioral limitation, not the proposed cause.
Counts as supported if: Representative models show unstable or absent relevant world representations, and controlled interventions establish that this deficiency causes their planning and reasoning failures.
Counts as contradicted if: Representative models exhibit stable relevant world representations and dependable planning and reasoning, or controlled studies show that the asserted causal explanation fails.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The planning study supports Marcus's concern about unreliable autonomous planning. It does not establish his causal explanation. The Othello study demonstrates learned representations in a narrower model, while RAP improves planning by changing world-state prediction, search, and rewards together. These findings neither establish nor decisively refute the broad causal claim.
Limitations
- Evidence supporting the behavioral component is not sufficient to score the entire causal proposition supported.
- Specialized synthetic models and externally assisted systems are not direct substitutes for standalone conversational LLMs.
- No numerical reliability threshold was supplied.
Evidence (3 items)
Supports the claim: Valmeekam and colleagues via arXiv,
approximately 34% in the Blocksworld dataset.
Why it matters here: The study supports the planning-reliability concern and reports severe deterioration after renaming domain elements. It also finds benefits from external verification; neither result directly measures the absence of stable internal world models.
- Publisher
- Valmeekam and colleagues via arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Li and colleagues via arXiv,
an emergent world representation.
Why it matters here: The researchers identify a board-state representation and intervene on it to change predictions. This challenges a blanket architectural absence claim, but the synthetic Othello model does not establish stable general world models in contemporary conversational LLMs.
- Publisher
- Li and colleagues via arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Hao and colleagues via arXiv,
33% relative improvement
Why it matters here: The framework uses an LLM for world-state prediction alongside tree search and rewards, improving planning performance. This is relevant to the proposed mechanism, but the combined intervention does not isolate world-model stability as the cause.
- Publisher
- Hao and colleagues via arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Controlled studies of representative 2023 models that measure world-state stability and independently manipulate it, then test whether planning and reasoning reliability changes while search, feedback, and computation are held comparable.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Hello, Multimodal Hallucinations
But in other ways, large language models are a profound step backwards; they lack stable models of the world, which means that they are poor at planning, and unable to reason reliably; everything becomes hit or miss..
- Where
- paragraph 24
- Published
- 2023-10-21
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-hello-multimodal-hallucinations-c3ae37-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-hello-multimodal-hallucinations-c3ae37-a1.