“Worse, they do not actually calculate the consequences of their actions at all.”
Our reading, the sentence that was checked:
LLMs do not calculate the consequences of their actions at all.
Assessment: Contradicted Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Humanity’s “Oh shit!” AI moment?, paragraph 6.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-humanitys-oh-shit-ai-moment-36db46-a1
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-12-12. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- do not actually calculate
- the consequences of their actions
- at all
Our interpretation
- The ordinary testable reading is that LLMs cannot work out how an action changes an outcome and use that consequence to guide their choice.
- The categorical phrase at all makes a controlled demonstration of consequence-sensitive choice sufficient to contradict this reading. It does not require reliable planning in every situation.
- This does not equate successful behavior with consciousness or a human-like internal model.
- The target is the December 2024 capability snapshot available before the December 12 post, using the current research reporting window.
Counts as supported if: Controlled tests across the relevant models find no ability to use action-outcome relationships, beyond superficial response patterns, to guide choices.
Counts as contradicted if: Controlled experiments show models selecting actions according to their predicted consequences, including changing behavior when the consequence of the same action changes.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Contradicted
The dated evidence goes against what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Models adjusted their performance to avoid a stated consequence, with the consequence reversed in the comparison condition. That contradicts the ordinary behavioral reading of not calculating consequences at all. A narrower claim about an unspecified internal mechanism remains unsettled.
Limitations
- The consequences were supplied explicitly in a simplified task.
- The result does not demonstrate human-like understanding or generally reliable foresight.
Evidence (1 item)
Goes against the claim: Apollo Research,
can strategically underperform
Why it matters here: The experiment reversed whether high or low scores would trigger unlearning and found consequence-sensitive underperformance. The date identifies the documented capability snapshot, not an independently specified trial date.
- Publisher
- Apollo Research
- Kind of source
- Independent measurement
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Humanity’s “Oh shit!” AI moment?
Worse, they do not actually calculate the consequences of their actions at all.
- Where
- paragraph 6
- Published
- 2024-12-12
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-humanitys-oh-shit-ai-moment-36db46-a1: Current. Outcome: Contradicted. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-humanitys-oh-shit-ai-moment-36db46-a1.