“Since we know that Imagen can draw images of horses riding astronauts (if the instructions are specific enough) we know that the failure to draw a horse riding an astronaut given the prompt “a horse riding an astronaut” can only have one explanation: the system doesn’t understand what the prompt “a horse riding an astronaut” means.”
Our reading, the sentence that was checked:
Since Imagen can draw horses riding astronauts when the instructions are specific enough, its failure to draw one from the prompt 'a horse riding an astronaut' can only be explained by the system not understanding what that prompt means.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Horse rides astronaut, paragraph 90.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-horse-rides-astronaut-63b833-a1
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-05-28. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- since Imagen can draw horses riding astronauts if the instructions are specific enough (qualifier, stated by the speaker)
From the speaker
- Imagen can draw images of horses riding astronauts
- if the instructions are specific enough
- can only have one explanation
- the system doesn’t understand what the prompt
- a horse riding an astronaut
- means
Our interpretation
- This concerns the original Imagen's May 2022 prompt failure.
- The success of more specific instructions is an asserted premise, not a future conditional forecast.
- Only one explanation commits to causal exclusivity. Evidence that some prompt changes help is insufficient unless competing causes of the original failure are excluded.
- The claim does not specifically identify a neural-network component, so understanding is assessed through preservation and use of the requested semantic roles.
Counts as supported if: Tests of the original Imagen verify the premise and establish failed semantic interpretation as the explanation of the original failure while excluding downstream conditioning, generation, sampling, and selection explanations.
Counts as contradicted if: Tests of the original Imagen establish another cause of the failure, including cases where the requested roles are represented correctly but the generation process fails to express them.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The premise is supported by Neyshabur's report that more specific wording improves the chance of success. The exclusive explanation is not established. Successful rewording does not by itself show where the original attempt failed. Research on other generators cannot supply the missing diagnosis for Imagen, and merely proposing alternatives would not justify calling Marcus contradicted.
Limitations
- The available demonstration is not a controlled causal experiment.
- An output-level failure cannot by itself distinguish absent semantic interpretation from failure to use an adequate representation.
- The post's acknowledged uncertainty limits confidence but does not erase its explicit exclusivity claim.
Evidence (2 items)
Supports the claim: Behnam Neyshabur,
helps increase the chance
Why it matters here: The firsthand report supports the premise that more specific instructions can produce the intended scene. It supplies no test establishing a unique cause for failures with the original wording.
- Publisher
- Behnam Neyshabur
- Kind of source
- Demonstration
- Describes events of
- Public from
Context only: Charles Lovering and Ellie Pavlick, arXiv,
stablediffusion 2.1 text-to-image model.
Why it matters here: This study relates training frequencies to success on reversed relationships. Because it tests another model, it cannot establish an alternative cause of the original Imagen failure.
- Publisher
- Charles Lovering and Ellie Pavlick, arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
A controlled investigation of the original Imagen that identifies the cause of the disputed failures and either excludes or demonstrates competing explanations.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Horse rides astronaut
Since we know that Imagen can draw images of horses riding astronauts (if the instructions are specific enough) we know that the failure to draw a horse riding an astronaut given the prompt “a horse riding an astronaut” can only have one explanation: the system doesn’t understand what the prompt “a horse riding an astronaut” means.
- Where
- paragraph 90
- Published
- 2022-05-28
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-horse-rides-astronaut-63b833-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.