“Behavioral similarity alone therefore provides no insight into whether the underlying processes support core components of general intelligence, such as judgment, error correction, or reliable generalization.”
Our reading, the sentence that was checked:
LLMs' behavioral similarity to humans alone gives no insight into whether they have judgment, error correction, or reliable generalization.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Rumors of AGI’s arrival have been greatly exaggerated, paragraph 26.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-rumors-of-agis-arrival-have-been-06f36e-a1
We recorded 18 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-02-17. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- Behavioral similarity alone
- provides no insight
- underlying processes
- judgment, error correction, or reliable generalization
Our interpretation
- The literal empirical reading is that resemblance to human outputs supplies no useful information about the specified underlying capabilities.
- This is stronger than saying that similar outputs do not prove identical mechanisms. I do not silently replace the stronger wording with that weaker proposition.
- A test would measure behavioral similarity separately from independently validated judgment, correction, or transfer, and ask whether similarity predicts those capabilities.
- For the contemporary assessment, I use the surrounding discussion's 2025 research period through the post date; older studies can supply context.
Counts as supported if: Well-designed comparisons find that behavioral similarity supplies no meaningful predictive information about independently measured underlying capabilities across relevant systems and tasks.
Counts as contradicted if: Reproducible comparisons show that behavioral similarity supplies useful predictive information about independently measured judgment, error correction, or generalization, even if it does not uniquely identify mechanisms.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The evidence supports caution about inferring mechanisms from matching outputs, but does not settle the stronger assertion that similarity provides no insight. Centaur offers potentially informative links between behavioral training and independent measurements; a subsequent evaluation also finds important failures. Neither result directly tests the claim's absolute wording. Scoring the weaker claim that resemblance is insufficient proof would change what Marcus said.
Limitations
- The strongest relevant studies use additional tests beyond behavioral similarity.
- The claim combines an empirical prediction question with a question about identifying internal mechanisms.
Evidence (2 items)
Context only: Nature; Binz et al.,
human neural activity
Why it matters here: Centaur links behavioral training with independent neural-alignment and transfer analyses, challenging an assumption that behavioral resemblance must be uninformative. The central findings appeared in the 2024 preprint, and the additional analyses mean this is not a direct test of similarity alone.
- Where
- Abstract
- Publisher
- Nature; Binz et al.
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Context only: arXiv; Namazova et al.,
human data.
Why it matters here: This evaluation shows that accurate behavioral prediction can coexist with important simulation failures. It supports caution about inference from resemblance, but does not establish that resemblance provides no information whatsoever.
- Where
- Abstract
- Publisher
- arXiv; Namazova et al.
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
A clear definition of 'no insight' and studies testing whether a separately measured degree of human-like behavior predicts independent measures of judgment, correction, and transfer.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Rumors of AGI’s arrival have been greatly exaggerated
Behavioral similarity alone therefore provides no insight into whether the underlying processes support core components of general intelligence, such as judgment, error correction, or reliable generalization.
- Where
- paragraph 26
- Published
- 2026-02-17
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-rumors-of-agis-arrival-have-been-06f36e-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-rumors-of-agis-arrival-have-been-06f36e-a1.