“great also to see OpenAI’s President Greg Brockman sorta kinda acknowledging the point I was making, in a rare note of realism from OpenAI.”
Our reading:
OpenAI President Greg Brockman sorta kinda acknowledged Marcus's point that code which compiles and passes the tests it was given is not the same as correct, secure, maintainable, well-architected software, and Marcus calls this a rare note of realism from OpenAI.
Assessment: Can’t be checked
What happened
Not graded on evidence
Outcome: Can’t be checked
An opinion, a recommendation, or too vague to test. Listed, not scored.
Evidence up to . Checked .
Why
Marcus offered a hedged interpretation of Brockman’s remarks, not a defined claim about the degree of agreement. TNW used similar acknowledgment language, which explains the context. But scoring Marcus would require choosing how much agreement counts as “sorta kinda” and what makes realism “rare.” Those standards are absent from his words. The claim therefore cannot fairly receive a factual verdict as written.
Caveats
- This does not establish that Marcus’s interpretation was wrong.
- The cited evidence is a news report; the primary interview page was excluded as PARTIAL.
- No separate verdict is proposed on the underlying distinction between passing tests and producing robust software.
Evidence (1 item)
Claims like this aren’t scored, so there is no evidence to show.
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- sorta kinda (their words)
- rare note of realism (evaluative) (their words)
Where we had to interpret
- The surrounding text identifies Marcus’s point as the distinction between passing supplied tests and producing correct, secure, maintainable, well-architected software.
- “Sorta kinda” leaves the degree and directness of acknowledgment unspecified. Requiring an explicit concession, or counting any admission of limitations, would introduce a threshold Marcus did not supply.
- “Rare note of realism” is an evaluative description without a defined comparison period or standard.
- The allowance for an ordinary reading of firm language does not justify converting this hedged assessment into a firm factual commitment.
We don’t grade claims like this. We only check the quote and who said it.
What would settle it
A more definite statement from Marcus identifying what acknowledgment he attributes to Brockman could support a separate factual assessment. Additional evidence alone cannot define the evaluative terms in this wording.
We found 4 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, “A model that produces code which compiles and passes the tests it was given is not the same as a model that produces correct, secure, maintainable, well-architected software”, paragraph 2.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2026-05-01.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, “A model that produces code which compiles and passes the tests it was given is not the same as a model that produces correct, secure, maintainable, well-architected software”
great also to see OpenAI’s President Greg Brockman sorta kinda acknowledging the point I was making, in a rare note of realism from OpenAI.
- Where
- paragraph 2
- Published
- 2026-05-01
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Can’t be checked. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.