“I rushed out a hot take on X, my usual cautions about how LLMs aren’t really AGI:”
Our reading, the sentence that was checked:
LLMs, including OpenAI's new o1, are not really AGI.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Gary Marcus vs Sam Altman on GPT-o1 aka Strawberry, paragraph 9.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-gary-marcus-vs-sam-altman-on-gpt-871f78-a2
We recorded 5 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-09-12. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- LLMs aren’t really AGI
- OpenAI’s latest, GPT o1
- Fast forward to today
Our interpretation
- Read AGI as broadly human-level, general intellectual capability, including reliable adaptation across unfamiliar tasks. This does not require perfection.
- Assess publicly available contemporary LLMs, including the newly released o1-preview, rather than every possible future LLM.
- The post explicitly concerns launch day, so the target is the capability state on September 12, 2024. Launch-era evaluations describe that state even where their individual testing dates are undisclosed.
Counts as supported if: Contemporary evaluations show substantial gaps in general task execution and adaptation inconsistent with broadly human-level general capability.
Counts as contradicted if: Credible broad evaluations establish that a contemporary LLM, including o1-preview, already possessed that general capability.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The launch-era system card, available September 17, 2024, reports substantial failures in biological agentic tasks across tested models, including o1-preview [e3]. Those practical execution gaps support the September 12 observation under the contract's reading of AGI as broadly human-level general capability [e3]. ARC Prize's September 13 report provides supporting context: o1-preview scored 21.2% on its public evaluation and 18% on its semi-private evaluation [e4]. A preprint published September 27 reports broad reasoning strengths, but describes progress toward AGI rather than establishing AGI [e5].
Limitations
- The system card tests several model checkpoints and configurations; it is not an exhaustive evaluation of all contemporary LLMs [e3].
- September 12 identifies the target capability state, not a known testing date for every system-card evaluation [e3].
- The later independent evaluations provide context rather than measurements confined to launch day [e4, e5].
- Support depends on the contract's broadly human-level interpretation of AGI.
Evidence (3 items)
Supports the claim: OpenAI,
biological agentic tasks.
Why it matters here: The launch-era card documents failures in computer-mediated tasks normally performed by human lab assistants; its METR section also reports difficulties using tools and environmental feedback. These are substantive capability gaps supporting the ordinary human-level reading, rather than evidence based merely on occasional errors.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: ARC Prize,
AGI.
Why it matters here: ARC Prize measured o1-preview at 21.2% on the public evaluation and 18% on the semi-private evaluation, alongside other contemporary LLMs. This supports the interpretation of limited adaptation while also documenting gains over GPT-4o; it is context because the tests extended beyond the target day.
- Publisher
- ARC Prize
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: Tianyang Zhong and colleagues / arXiv,
intricate reasoning and knowledge integration
Why it matters here: This evaluation documents strengths across multiple domains, cautioning against portraying o1-preview as narrowly competent or incapable of reasoning. It describes progress toward AGI, and its later testing is not a measurement confined to launch day.
- Publisher
- Tianyang Zhong and colleagues / arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Gary Marcus vs Sam Altman on GPT-o1 aka Strawberry
I rushed out a hot take on X, my usual cautions about how LLMs aren’t really AGI:
- Where
- paragraph 9
- Published
- 2024-09-12
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-gary-marcus-vs-sam-altman-on-gpt-871f78-a2: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-gary-marcus-vs-sam-altman-on-gpt-871f78-a2.