“We know nothing about the failure rate. We know nothing about the training/post training/data augmention.”
Our reading:
OpenAI disclosed nothing about the failure rate or the training, post-training, or data augmentation of the system behind its math results.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
OpenAI’s September 10, 2026 update says its Navier–Stokes model used large-scale reinforcement learning on a previously pretrained model [e4]. Resolving the claim requires evidence about disclosures available on October 6–7, including a demonstrated connection between that training description and the October model.
Caveats
- The supplied evidence does not establish whether the September training disclosure applies to the October model.
- No supplied evidence item documents the October release’s failure-frequency disclosures.
- These gaps do not establish that relevant information was absent from the public record.
Evidence (1 item)
Context only: OpenAI,
reinforcement
Why it matters: The dated update disclosed a substantive training method for the earlier Navier–Stokes model. It is relevant counterevidence to a broad claim of total ignorance, but does not independently identify the October collection's model version or training.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- Assess two components: public information about failure frequency and public information about training, post-training or data augmentation.
- Failure rate requires a defined outcome and denominator. Counts of published manuscripts, result families and attempted problems are not automatically comparable.
- Training information means substantive information about methods or data, rather than merely a training start date.
- Assess what was public at the October 6 release and when Marcus wrote on October 7. Earlier training disclosures need a demonstrated connection to the relevant model.
Right if: The contemporaneous record provides neither interpretable failure-frequency information nor substantive information about the relevant model's training, post-training or data augmentation.
Wrong if: The contemporaneous record provides both interpretable failure-frequency information and substantive training information about the relevant model. If only one component is established false while the other is established true, the combined claim is mixed.
We wrote this test after the outcome was known.
What would settle it
A sufficiently complete record of disclosures available by Marcus’s October 7 publication. It must establish whether substantive training information applied to the October model and whether failure-frequency information defined outcomes, the attempted population, selection rules and verification status.
We found 7 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Brief remark on OpenAI’s giant math drop, paragraph 15.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2026-10-07.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Brief remark on OpenAI’s giant math drop
We know nothing about the failure rate. We know nothing about the training/post training/data augmention.
- Where
- paragraph 15
- Published
- 2026-10-07
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
The final check failed: a cited source didn’t say what the write-up said.