Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts
We recorded 10 claims from this post. 7 were checked against evidence. 3 can’t be checked. 4 more are held. Read the post at its source.
Claims from this post (10)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
“In particular, DALL-E2 was very poor at following specifications about parts of objects.”
What we tested: As of October 2022, DALL-E2 was very poor at following prompt specifications about parts of objects.
Assessment: Can’t be checkedSaid
“One plausible explanation is that OpenAI has put a lot more work, with data augmentation and other such techniques, into math word problems than into image generation.”
What we tested: One plausible explanation for the difference in improvement is that OpenAI has put a lot more work, using data augmentation and other such techniques, into math word problems than into image generation.
Assessment: ContradictedSaid
“The remaining thirteen it got wrong in all three attempts.”
What we tested: In the authors' 17 part-specification experiments, ChatGPT with DALL-E3 got 13 wrong in all three attempts.
What happened: The evaluator accepts the umbrella table in addition to the deer, wooden-legged table, cat, and gold airplane. Those five accepted cases leave at most twelve failures among seventeen. (Ernest Davis, NYU-hosted experimental record, )
Assessment: UnresolvedSaid
“We did not see anything like the kind of improvement that ChatGPT has exhibited over math word problems, over the last two years.”
What we tested: In the authors' body-parts image experiments, they did not see anything like the kind of improvement that ChatGPT has shown on math word problems over the last two years.
Assessment: UnresolvedSaid
“The third illustrates that it cannot reliably eliminate an object part.”
What we tested: ChatGPT with DALL-E 3 cannot reliably eliminate a part of an object from an image.
Assessment: UnresolvedSaid
“The first example illustrates that ChatGPT with DALL-E 3 is unable to reliably associate labels correctly with the parts of an object.”
What we tested: ChatGPT with DALL-E 3 cannot reliably associate labels correctly with the parts of an object.
Assessment: UnresolvedSaid
“The second example illustrates that it cannot reliably produce a non-standard number of body parts.”
What we tested: ChatGPT with DALL-E 3 cannot reliably produce images with a non-standard number of body parts.
Assessment: UnresolvedSaid
“But current technology has proven quite poor at such sanity checks. New technology may be needed.”
What we tested: Current AI technology is quite poor at checking whether its own image output satisfies the prompt and feedback, and new technology may be needed.
Assessment: Can’t be checkedSaid
“Relating language and the visual world robustly may require altogether new approaches to learning language.”
What we tested: Robustly relating language and the visual world may require altogether new approaches to learning language.
Assessment: Can’t be checkedSaid
“Until then the foundations of AI remain unsteady.”
What we tested: Until new approaches robustly relate language and vision, the foundations of AI remain unsteady.
Held from this post (4)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: In current generative AI approaches, the relation between language and images remains superficial and unpredictable, with little guarantee any trial is correct.
- The record does not meet the current rules. Our reading, not assessed: Compositional and part-related errors remain common in generative AI, with default answers often reflecting a lack of full comprehension of language or the world.