“The third illustrates that it cannot reliably eliminate an object part.”
Our reading, the sentence that was checked:
ChatGPT with DALL-E 3 cannot reliably eliminate a part of an object from an image.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts, paragraph 28.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-dont-ride-this-bike-generative-ais-06f3e3-a2
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-12-08. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- based on a small number of examples (qualifier, stated by the speaker)
From the speaker
- cannot reliably eliminate an object part
- a violin with strings but no bridge
- Second and third prompts returned essentially identical results
Our interpretation
- It refers to ChatGPT with DALL-E 3.
- The plain reading covers generating an object with a specified usual part absent, including renewed generation after corrective feedback. It does not require pixel-preserving image editing.
- The target period is November 2024, as specified for the underlying experiments.
- Excluding an entire object from a scene is a related but different task.
Counts as supported if: November 2024 records show repeated retention of an explicitly excluded object part despite clear omission or correction instructions.
Counts as contradicted if: Comparable tests show the requested parts consistently absent while retaining the rest of the object and satisfying the other material instructions.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The SPICE paper's Figure 9 describes repeated failures to generate a bridge-free violin in January 2025 [e3]. Its November 2024 statement cites the original Davis experiment rather than a separate test [e3]. This evidence does not settle part-omission reliability in the specified November 2024 period [e3].
Limitations
- The packet contains no independently verified November 2024 omission outputs or directly relevant contemporary evaluation.
- The later results do not establish that the January deployment was unchanged from November.
Evidence (1 item)
Context only: arXiv; Kenan Tang, Yanhong Li, and Yao Qin,
SPICE can generate content that DALL·E 3 cannot.
Why it matters here: Figure 9 describes repeated failures involving a bridge-free violin and a four-eared rabbit by January 2025. This is closely relevant later evidence, but the paper's November statement cites the original Davis record rather than a separate November test.
- Publisher
- arXiv; Kenan Tang, Yanhong Li, and Yao Qin
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
An independent audit of November 2024 prompts and inspectable images, covering all repeated attempts and checking that the specified part is absent while the rest of the object and other required features remain.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
The third illustrates that it cannot reliably eliminate an object part.
- Where
- paragraph 28
- Published
- 2024-12-08
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-dont-ride-this-bike-generative-ais-06f3e3-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-dont-ride-this-bike-generative-ais-06f3e3-a2.