“That’s where the deep understanding of language should live—but, despite the PR, it isn’t there.”
Our reading, the sentence that was checked:
Imagen lacks deep understanding of language despite Google's PR.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Horse rides astronaut, paragraph 83.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-horse-rides-astronaut-4f1bbd-a1
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-05-28. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- deep understanding of language
- despite the PR
- it isn’t there
Our interpretation
- In this context, the ordinary reading is reliable understanding of who does what to whom in compositional image instructions.
- This is a behavioral claim about Imagen as a complete system, not a claim about consciousness or the absence of every language capability.
- I use the original model's public evaluation window, from its May 23 announcement through the May 28 post, because the sentence describes an ongoing capability without specifying a reporting period.
- Neither occasional mistakes nor occasional successes alone settle the claim; evidence should show a recurring weakness or robust success on the relevant instruction types.
Counts as supported if: Evaluations of the original Imagen show recurring failures to preserve basic roles or relationships in compositional instructions, inconsistent with reliable understanding in the sense discussed.
Counts as contradicted if: Controlled evaluations show reliable handling of the relevant role reversals, relationships, and equivalent paraphrases, with the cited failures shown to be isolated exceptions.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Under the contextual reading of reliable compositional instruction understanding, the original paper supports Marcus: its authors acknowledge alignment difficulties in the category containing the disputed prompt. Imagen's strong comparative results are material qualifications, but outperforming other models does not demonstrate reliable handling of these instructions. This finding does not establish that Imagen had no language capabilities or identify the internal cause of its failures.
Limitations
- The result depends on interpreting deep understanding through the role-and-relationship behavior discussed in the post.
- The evidence is the authors' qualitative category assessment, not a comprehensive paraphrase robustness study.
- It applies to the original Imagen, not subsequent versions.
Evidence (2 items)
Supports the claim: Google Research, Brain Team; copy hosted by 3DVAR,
struggle generating well aligned images
Why it matters here: The original authors identify a category-level alignment weakness in the figure containing the horse-riding-an-astronaut prompt. This supports the contextual reading that reliable compositional instruction understanding was missing.
- Publisher
- Google Research, Brain Team; copy hosted by 3DVAR
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Google Research,
Human raters strongly prefer Imagen
Why it matters here: The favorable comparative evaluation cautions against interpreting Marcus as showing no useful language capability. Preference over competing systems does not establish reliable understanding of the specific relationships at issue.
- Publisher
- Google Research
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Horse rides astronaut
That’s where the deep understanding of language should live—but, despite the PR, it isn’t there.
- Where
- paragraph 83
- Published
- 2022-05-28
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-horse-rides-astronaut-4f1bbd-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.