“And that might be surprising, but it turns out that the way that these prediction machines work is they follow correlations rather than causation.”
Our reading:
LLM prediction machines fail to follow instructions because they follow correlations rather than causation.
Assessment: Unresolved
What happened
Does the evidence support the cause?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The retrieved studies investigate relevant mechanisms, but neither establishes Marcus's proposed causal link. Instruction following appears to involve several task-dependent skills. Separate work finds both spurious correlations and improved causal patterns after additional training. These findings neither prove nor rule out correlation reliance as a cause of the specific instruction failures.
Caveats
- Both cited studies are preprints.
- Their interventions and outcome measures do not directly match the claimed causal explanation.
- A training objective cannot establish the absence of learned causal representations.
Evidence (2 items)
Context only: Elisabetta Rocchetti and Alfio Ferrara, arXiv,
sparse asymmetric dependencies
Why it matters: The study finds task-dependent organization of instruction-following skills in three models. Its ablations concern representations and diagnostic probes; they do not establish that failure to represent causation causes the instruction violations Marcus describes.
- Publisher
- Elisabetta Rocchetti and Alfio Ferrara, arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Context only: Zhizhang Fu and colleagues, arXiv,
reduces spurious correlations
Why it matters: The authors report causal weaknesses in some reasoning processes and stronger causal patterns after reinforcement learning with verifiable rewards. This complicates the blanket correlation-versus-causation description, but does not directly test the claimed explanation for instruction failures.
- Where
- Abstract
- Publisher
- Zhizhang Fu and colleagues, arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
How we judged it
- Kind of claim
- Causal claim
- Deadline
- None given. We don’t add one.
Where we had to interpret
- In context, Marcus offers dependence on correlations rather than causal understanding as an explanation for instruction-following failures.
- This is assessed as a causal explanation, not merely a description of a prediction-based training objective.
- The claim does not specify a mechanism, intervention, or proportion of failures attributable to the proposed cause.
Right if: Controlled studies distinguish reliance on superficial correlations from causal understanding and demonstrate that this reliance causes the relevant instruction violations.
Wrong if: Comparable controlled studies rule out the proposed mechanism or show that the alleged causal limitation does not explain instruction failures.
We wrote this test after the outcome was known.
What would settle it
Controlled experiments on contemporary models that manipulate misleading correlations and causal structure, measure explicit instruction violations, and distinguish this mechanism from competing explanations.
We found 32 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Report from the Vatican, paragraph 23.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-11-09. The post may have been edited after 2026-10-31.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Report from the Vatican
And that might be surprising, but it turns out that the way that these prediction machines work is they follow correlations rather than causation.
- Where
- paragraph 23
- Published
- 2026-10-31
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-11-09.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.