“Mollick’s notion of “jagged” AI is exactly what I used to mean by the term “pointillisitic” and what I predicted about o3 in November. AI that works some of the time but not always, markedly better on some problems than others.”
Our reading, the sentence that was checked:
o3 works only some of the time and is markedly better on some problems than others, as Marcus predicted in November 2024.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, OpenAI’s o3 and Tyler Cowen’s Misguided AGI Fantasy, paragraph 20.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-openais-o3-and-tyler-cowens-misguided-9a20fa-a1
We recorded 4 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-04-17. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- what I predicted about o3 in November
- works some of the time but not always
- markedly better on some problems than others
Our interpretation
- The plain performance claim is that launch-era o3 combines substantial strengths with important task-dependent weaknesses.
- Works some of the time does not specify an overall success percentage or mean that most answers fail.
- The November attribution is a separate factual component requiring a dated contemporaneous record; November is read as November 2024.
- The target period covers the ongoing performance observation at release. Earlier prediction records are treated as historical context and checked separately.
- This is an observation plus a retrospective attribution, not a new forecast.
Counts as supported if: Launch-era evaluations establish pronounced strengths and weaknesses, and a contemporaneous November 2024 record establishes the attributed prediction.
Counts as contradicted if: Evaluations undermine the asserted unevenness, or a sufficiently complete historical record establishes that the November attribution is wrong. Finding a later prediction alone does not prove none existed earlier.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The performance component is supported: OpenAI's launch records show strong results in several domains alongside substantial factual-answer failures. The complete claim remains unresolved because the November attribution was not verified. A dated December 20 record contains a similar prediction, but that neither establishes November nor proves that no earlier prediction existed.
Limitations
- The performance evidence is company-reported.
- Different benchmarks measure different tasks and cannot be combined into one accuracy figure.
- Failure to locate an earlier statement is not evidence that it never existed.
Evidence (3 items)
Supports the claim: OpenAI,
new SOTA
Why it matters here: The launch disclosure documents strong coding and multimodal performance. Together with the system card's factual-answer failures, it supports the performance component of the claim.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: OpenAI,
inaccurate/hallucinated claims.
Why it matters here: The accompanying table reports o3 accuracy of 0.49 on SimpleQA and 0.59 on PersonQA, with substantial hallucination rates. These documented weaknesses coexist with its strong benchmark results; they do not provide a universal everyday-use accuracy rate.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: Techmeme,
less reliably
Why it matters here: The dated archive preserves a similar prediction in December. It establishes neither the claimed November date nor the truth of the prediction, and does not exclude an earlier statement.
- Publisher
- Techmeme
- Kind of source
- Secondary summary
- Describes events of
- Public from
What would settle it
A verifiable November 2024 post, recording, or archived statement containing the attributed prediction would settle the missing historical component.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, OpenAI’s o3 and Tyler Cowen’s Misguided AGI Fantasy
Mollick’s notion of “jagged” AI is exactly what I used to mean by the term “pointillisitic” and what I predicted about o3 in November. AI that works some of the time but not always, markedly better on some problems than others.
- Where
- paragraph 20
- Published
- 2025-04-17
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-openais-o3-and-tyler-cowens-misguided-9a20fa-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-openais-o3-and-tyler-cowens-misguided-9a20fa-a1.