“In particular, DALL-E2 was very poor at following specifications about parts of objects.”
Our reading, the sentence that was checked:
As of October 2022, DALL-E2 was very poor at following prompt specifications about parts of objects.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts, paragraph 6.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-dont-ride-this-bike-generative-ais-7975a8-a1
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-12-08. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- as of October 2022 (qualifier, stated by the speaker)
From the speaker
- DALL-E2
- very poor at following specifications about parts of objects
- October 2022
Our interpretation
- The ordinary reading is that explicit requirements about object parts frequently failed across varied prompts.
- Success requires satisfying the specified part attributes, counts, substitutions, or omissions. General image quality does not establish success.
- October 2022 is the observation period supplied by the surrounding passage.
Counts as supported if: Contemporaneous, independently assessed results show frequent failures across varied prompts specifying object parts.
Counts as contradicted if: Comparable October 2022 results show that DALL-E2 usually satisfied explicit object-part requirements.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Independent research documents related compositional weaknesses, but it concerns different tasks and an earlier period. The original October record was located, but its text and the authors' conclusions do not independently establish the claimed performance. The evidence therefore does not settle this specific observation.
Limitations
- This is an evidence gap, not evidence that DALL-E2 performed well.
- The assessment does not generalize results about object relations to object parts.
Evidence (1 item)
Context only: arXiv; Colin Conwell and Tomer Ullman,
~22% of images matched basic relation prompts.
Why it matters here: This documents difficulty with relations between objects. It does not measure October 2022 performance on specifications about object parts.
- Where
- Abstract
- Publisher
- arXiv; Colin Conwell and Tomer Ullman
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
An independent audit of the dated October outputs, or a contemporaneous evaluation covering varied object-part specifications.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
In particular, DALL-E2 was very poor at following specifications about parts of objects.
- Where
- paragraph 6
- Published
- 2024-12-08
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-dont-ride-this-bike-generative-ais-7975a8-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-dont-ride-this-bike-generative-ais-7975a8-a1.