Horse rides Astronaut, redux
We recorded 5 claims from this post. 4 were checked against evidence. 1 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (5)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Observation
“DALL-E 2 is, I dare say, not as smart as it seems.”
Our reading: DALL-E 2 is not as smart as it appears.
No horizon
Assessment: UnresolvedSaid
Causal claim
“In other words, the problem was with its language understanding, rather than with illustration per se. The model was perfectly capable of drawing a horse riding an astronaut, it just didn’t know that the phrase horse rides astronaut was a request for a horse riding an astronaut.”
Our reading: DALL-E's failure on 'horse rides astronaut' was due to deficient language understanding, not illustration ability.
No horizon Evidence as of
Limitation: The original images and complete sampling records were not available in the retrieved text.
Assessment: UnresolvedSaid
Observation
“One winner out of three to be exact.”
Our reading: After several retries, Gemini produced only one correct horse-riding-astronaut image out of three.
No horizon Evidence as of
Limitation: This concerns one historical response, not Gemini's general success rate.
Assessment: UnresolvedSaid
Observation
“I think we can safely reach the same conclusion as before. Gemini can draw the desired image. But it’s still like pulling teeth to get there.”
Our reading: Gemini can draw a horse riding an astronaut, but getting it to do so is very difficult.
No horizon Evidence as of
Limitation: A verified difficult session would support the local observation more strongly than a general claim about typical performance.
Assessment: ContradictedSaid
Observation
“In some ways we have made progress, but in the ones I keep harping on—factuality and compositionality—we have not.”
Our reading: Generative AI has made no progress on factuality and compositionality since 2022, despite progress in other respects.
No horizon Evidence as of
Limitation: The verdict depends on reading progress as measurable improvement rather than complete resolution.
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.