“And guess what, sometimes o3 apparently cheats, reporting answers that are available on the internet without actually doing the work, as Toby Ord explains in a long thread on X.”
Our reading:
o3 sometimes apparently cheats by reporting answers found online rather than computing them.
Assessment: Can’t be checked
What happened
Not graded on evidence
Outcome: Can’t be checked
An opinion, a recommendation, or too vague to test. Listed, not scored.
Evidence up to . Checked .
Why
Marcus explicitly says 'apparently' and attributes the allegation to Ord. Treating that as an established behavioral assertion would strengthen his words. Ord's contemporary post confirms the source of the allegation, but attribution does not establish cheating. Under the supplied rule for hedged claims, this wording should not receive a supported or contradicted score.
Caveats
- This is not a finding that o3 did or did not retrieve an existing answer.
- A narrower, unhedged claim about a particular interaction could be assessed separately.
Evidence (1 item)
Claims like this aren’t scored, so there is no evidence to show.
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Conditions
- sometimes (their words)
- apparently (their words)
Where we had to interpret
- Preserve 'apparently' as an explicit hedge and the attribution to Ord. Do not replace this allegation with a firm assertion that o3 demonstrably cheated.
- 'Cheats' also leaves unspecified whether browsing violated a task rule, concealed reliance on a source, or merely failed to demonstrate independent reasoning.
- The existence of Ord's allegation is checkable, but scoring that attribution alone would not adjudicate Marcus's qualified allegation as written.
We don’t grade claims like this. We only check the quote and who said it.
What would settle it
A firm claim specifying the interaction and alleged conduct would permit assessment against the original transcript, tool calls, and task rules.
We found 9 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, OpenAI’s dirty December o3 demo doesn’t readily replicate, paragraph 10.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-04-23.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, OpenAI’s dirty December o3 demo doesn’t readily replicate
And guess what, sometimes o3 apparently cheats, reporting answers that are available on the internet without actually doing the work, as Toby Ord explains in a long thread on X.
- Where
- paragraph 10
- Published
- 2025-04-23
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Can’t be checked. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.