“The John Locke hypothesis that you can learn physics purely from sense data is failing, over, and over, and over again; this time perhaps (if I had to guess) to the tune of a billion dollars.”
Our reading:
The idea that AI can learn physics purely from sense data keeps failing, and Sora is the latest example.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The November 4, 2024 study reports failures on unfamiliar physical situations, alongside perfect generalization within the training distribution and measurable scaling for new combinations [e3]. This supports a limited finding about the tested models, but does not establish repeated failures of the general sensory-learning hypothesis [e3]. Sora's December 9, 2024 system card describes generated text captions for visual training data, so it does not establish that Sora learns exclusively from sensory inputs [e4].
Caveats
- The controlled study covers a restricted set of models and physical situations. It does not establish the general limits of sensory learning or evaluate Sora [e3].
- The evidence does not establish a sequence of qualifying failures that includes Sora [e3] [e4].
- The training disclosure does not settle whether Sora fits the claim's meaning of 'purely from sense data' [e4].
Evidence (2 items)
Supports the claim: arXiv; Bingyi Kang and colleagues,
failure in out-of-distribution scenarios.
Why it matters: Controlled video-learning experiments support the narrower claim that physical generalization remained unreliable. The same result reports successful learning within the training distribution, limiting any inference that sensory learning simply fails.
- Publisher
- arXiv; Bingyi Kang and colleagues
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Context only: OpenAI,
captions
Why it matters: Sora's documented training uses generated language alongside visual data. Its failures therefore do not constitute a clean test of learning exclusively from sensory inputs, although they remain relevant to learning without explicit physical rules.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- Read as an observation about repeated attempts available by December 2024, rather than a forecast that sensory learning can never succeed.
- Learning physics means acquiring reliable physical behavior that generalizes beyond familiar examples; it does not require learning every physical law.
- The February-to-December Sora discussion supplies the observation window, although the wider history of repeated attempts is unspecified.
- The billion-dollar aside is expressly a guess. It is not treated as a firm spending claim or included in the outcome.
- Purely from sense data could mean exclusively sensory training inputs, or learning without explicit physical rules. Those readings require different evidence.
Right if: Training descriptions and evaluations establish repeated failures to acquire reliable, generalizable physics from sensory observations, including a Sora training setup that fits the claimed sensory-learning approach.
Wrong if: Comparable sensory-learning systems demonstrate robust physical generalization across varied unfamiliar situations, or evidence establishes that the asserted repeated failures did not occur.
We wrote this test after the outcome was known.
What would settle it
Define the permitted training inputs and assumptions, and a threshold for reliable physical generalization. Then provide training records and controlled evaluations for systems available from February 1 through December 10, 2024, including Sora. These must establish repeated qualifying failures or robust physical generalization across varied unfamiliar situations.
We found 7 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Sora still appears to have trouble with physics, paragraph 11.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2024-12-10.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Sora still appears to have trouble with physics
The John Locke hypothesis that you can learn physics purely from sense data is failing, over, and over, and over again; this time perhaps (if I had to guess) to the tune of a billion dollars.
- Where
- paragraph 11
- Published
- 2024-12-10
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
The final check failed: a cited source didn’t say what the write-up said.