“All three examples show the difficulty that the system has in associating user feedback with parts of images.”
Our reading:
ChatGPT with DALL-E 3 has difficulty associating user feedback with parts of images.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The study available on May 14, 2024 reports prompt refinement over three iterations using GPT-4V feedback with DALL-E 3 [e4]. It tests a custom automated workflow on attribute and relation prompts and reference-image recreation [e4]. These results do not establish whether ordinary ChatGPT successfully applied the claimed November 2024 corrections to image parts [e4].
Caveats
- The original November 2024 before-and-after images were not independently verified.
- The packet contains no comparable November test of part-specific feedback in ordinary ChatGPT.
- Success in a custom automated workflow does not contradict a claim of difficulty in ordinary interaction.
- The evidence does not establish an internal cause of correction failures.
Evidence (1 item)
Context only: CVPR 2024; Shihong Liu and colleagues, version hosted on arXiv,
using a few (three) iterations.
Why it matters: The study reports useful iterative refinement through a custom GPT-4V and DALL-E 3 workflow. This qualifies any suggestion that feedback is universally ineffective, but does not test the three November ChatGPT interactions; the exact experiment dates are unspecified.
- Publisher
- CVPR 2024; Shihong Liu and colleagues, version hosted on arXiv
- Kind of source
- Peer-reviewed paper
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- The plain reading is behavioral: the system repeatedly fails to apply clear corrections about particular image parts. It does not establish a specific internal cognitive mechanism.
- The three examples are bicycle labels, rabbit ears, and the violin bridge.
- The target period is November 2024, when the post says these interactions occurred.
- Difficulty permits occasional successful corrections; it does not mean feedback never helps.
Right if: Verified interaction sequences show that clear feedback about the relevant parts is followed by images retaining the specified errors across the three examples or closely comparable repeated tests.
Wrong if: The complete interaction records show that the specified corrections were successfully applied, or directly comparable testing establishes that the apparent failures resulted from misdescription or scoring errors.
We wrote this test after the outcome was known.
What would settle it
Independent inspection of the complete November 2024 conversations and images, or comparable dated tests in the same ChatGPT deployment, showing whether the named part errors persisted after clear feedback.
We found 11 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts, paragraph 28.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2024-12-08.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
All three examples show the difficulty that the system has in associating user feedback with parts of images.
- Where
- paragraph 28
- Published
- 2024-12-08
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
The final check failed: a cited source didn’t say what the write-up said, and a condition they stated was left out.