Race, statistics, and the persistent cognitive limitations of DALL-E
We recorded 7 claims from this post. 2 were checked against evidence. 5 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (7)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“What one really needs, and no one yet knows how to build, is a system that can derive semantics of wholes from their parts as a function of their syntax.”
Our reading: AI needs systems that derive the meaning of wholes from their parts according to syntax, which no one yet knows how to build.
No horizon
Assessment: UnresolvedSaid
Causal claim
“About 18 months have passed; although there are certainly improvements (the images are vastly better), owing presumably in large part to a larger training set, the fundamental approach (mapping words to images, rather than constructing intermediate models of the world and how language and images relate to those models) remains the same, and so the core issue remains.”
Our reading: About 18 months after DALL-E 2, the images are vastly better, presumably largely because of a larger training set, but the fundamental approach of mapping words to images rather than building intermediate world models is the same, and so the core compositionality issue remains.
No horizon Evidence as of
Limitation: The technical paper was read through a third-party reproduction because the original PDF exceeded the reader's size limit.
Assessment: Can’t be checkedSaid
Observation
“It’s that it can’t separate the world from the statistics of its dataset. DALL-E does not have a cognitive construct of a doctor or a patient or a human being or an occupation or medicine or race or egalitarianism or equal opportunity or any of that.”
Our reading: DALL-E 3 cannot distinguish the world from its training data statistics and has no cognitive concepts of doctors, patients, race, or similar.
No horizon
Assessment: UnresolvedSaid
Causal claim
“It’s the same issue of canonical versus noncanonical statistics, all over again.”
Our reading: Multimodal ChatGPT's trouble telling time from watch images comes from the same canonical-versus-noncanonical statistics problem seen in DALL-E.
No horizon Evidence as of
Limitation: Later model evaluations cannot be substituted for the historical deployment.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Blind fealty to the statistics of arbitrary data is not AGI. It’s a hack. And not one that we should want.”
Our reading: Blind fealty to the statistics of arbitrary data is not AGI; it is a hack, and not one we should want.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Our focus should be on figuring on how to build AI that can represent and reason about *values*, rather than simply perpetuating past data”.”
Our reading: AI research should focus on building systems that can represent and reason about values instead of just perpetuating past data.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“But the level of actual cognitive comprehension of here is low. The modest number of properly labeled examples is presumably swamped by all the billions of watch advertisements.”
Our reading: Multimodal ChatGPT has low real comprehension, and the few correctly labeled clock examples are probably drowned out by billions of watch ads showing the canonical time.
No horizon
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.