“The point is that you can’t count on the system to correctly interpret any given prompt, because the language understanding still is far from perfect. Even three years later.”
Our reading:
Even three years later, you can't count on Imagen to correctly interpret any given prompt, because its language understanding is still far from perfect.
Assessment: Unresolved
What happened
Does the evidence support the cause?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
Google acknowledges limitations consistent with Marcus’s reliability concern. That does not settle his explanation of their cause. The retrieved evidence does not distinguish deficient language interpretation from failures elsewhere in the generation process for the relevant June system. Neither the causal claim nor its negation is established.
Caveats
- The model card is contextual evidence, not a causal experiment.
- High benchmark performance would not by itself refute failures on unusual prompts.
- Another contributing cause would not exclude a language-understanding contribution.
Evidence (1 item)
Context only: Google DeepMind,
While Imagen 4 and other current strong models achieve impressive performance, they still exhibit shortcomings in certain capabilities.
Why it matters: Google acknowledges Imagen 4 limitations and discusses difficult reasoning and compositional tasks. This predates the target reporting window and does not establish that deficient language understanding caused the failures described in June.
- Publisher
- Google DeepMind
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
Probably right?
One AI thinks this is probably right, and the other says it’s too early to tell. The evidence isn’t conclusive, so the verdict above stays unresolved. It counts only when you pick Speculative under “How sure?”.
- Anthropic’s AI: Google's own Imagen 4 model card says compositional, spatial and complex-language prompts are still among the hardest. That admission supports his main point that prompt interpretation can't be relied on, though it doesn't isolate the cause.
How we judged it
- Kind of claim
- Causal claim
- Deadline
- None given. We don’t add one.
Conditions
- even though fiddling with prompts can get you what you want (their words)
Where we had to interpret
- The immediate referent is Imagen, represented by Imagen 4 in the current tests. Results from other models cannot settle this claim.
- The ordinary reading is that users cannot depend on correct interpretation across clear prompts, including unusual combinations and relationships, and that shortcomings in language understanding contribute to the failures.
- No numerical failure-rate threshold is imposed. The claim does not require every prompt to fail or language understanding to be the sole cause.
- Language understanding is assessed through handling requested meanings, attributes, and relationships, without requiring a particular internal architecture.
- The target is the current June 2025 state through publication. June 1–25 is used as the reporting window because no formal measurement period is named.
Right if: Evidence about the contemporaneous Imagen 4 system shows meaningful failures to follow clear prompts and establishes that deficient interpretation of requested meanings or relationships contributes to those failures. Suitable evidence could include controlled prompt contrasts and diagnostic interventions tied to the interpretation failure.
Wrong if: Evidence about the same Imagen 4 system establishes dependable interpretation across the relevant prompt types and refutes the claimed contribution from language-understanding failures. Identifying an additional cause, or showing isolated successes, does not suffice.
We wrote this test after the outcome was known.
What would settle it
Controlled tests of the June 2025 Imagen 4 system that measure prompt adherence and establish or exclude interpretation failures as a contributing cause.
We found 8 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Image generation: Still crazy after all these years, paragraph 39.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-06-25.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Image generation: Still crazy after all these years
The point is that you can’t count on the system to correctly interpret any given prompt, because the language understanding still is far from perfect. Even three years later.
- Where
- paragraph 39
- Published
- 2025-06-25
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.