Sora Can’t Handle the Truth
We recorded 6 claims from this post. 3 were checked against evidence. 3 can’t be checked. 4 more are held. Read the post at its source.
Claims from this post (6)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Causal claim
“It’s probably not fair to blame the weird board on a lack of data.”
Our reading: Sora's erroneous chessboard is probably not due to a lack of training data.
No horizon
Assessment: UnresolvedSaid
Observation
“But it’s drawing the wrong generalization.”
Our reading: Sora arguably shows some generalization, but in the monkey-chess case it is drawing the wrong generalization.
No horizon Evidence as of
Limitation: The evidence packet has no independent description or analysis of the original monkey-chess video.
Assessment: UnresolvedSaid
Observation
“Sora fundamentally revolves around trying to take pixels and reconstruct what pixels might be nearby in space and time. The forest is never perceived apart from the trees, and the chessboard is never perceived apart from its constituent squares.”
Our reading: Sora fundamentally works by reconstructing nearby pixels in space and time rather than perceiving whole objects.
No horizon Evidence as of
Limitation: The cited review appeared after the post and is used only as context about the original model.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“From AGI perspective, however, this is an immense warning sign.”
Our reading: From an AGI perspective, Sora's fixation on the local at the expense of a deep representation of the global is an immense warning sign.
No horizon
Scope: from an AGI perspective
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The sooner the AI industry -- and the world at large — recognizes this, the better.”
Our reading: The AI industry and the world should recognize soon that generative AI shares a persistent factuality and comprehension problem.
No horizon
Assessment: SupportedSaid
Observation
“Gary Marcus is astonished that some people still haven’t recognized that generative AI has struggled mightily when it comes to world models, high level reasoning and factuality, even as evidence for that conclusion continues to pour in, even with data of ever-greater magnitude and sophistication..”
Our reading: Generative AI has struggled mightily with world models, high-level reasoning and factuality, even with ever more data.
No horizon Evidence as of
Limitation: The severity term is qualitative, so the outcome depends partly on the stated reading.
Held from this post (4)
These records did not pass every check, so nothing about them is published. Held is not a finding.