“So for the most part, we’ve been stuck in a world in which all the major tech companies are building exactly the same thing (bigger and bigger LLMs based on more data), achieving almost exactly the same results (a passel of GPT-4 level models, hardly different from each other, all struggling with hallucinations and boneheaded errors), and hardly any investment in anything else.”
Our reading, the sentence that was checked:
Marcus says that, for the most part, all the major tech companies are building the same thing (bigger LLMs on more data) and getting almost the same results (GPT-4-level models that barely differ and all struggle with hallucinations and boneheaded errors), with hardly any investment in anything else.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic, paragraph 51.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-alphaproof-alphageometry-chatgpt-bb6581-a1
We recorded 17 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-07-28. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- for the most part (scope, stated by the speaker)
From the speaker
- for the most part
- all the major tech companies
- bigger and bigger LLMs based on more data
- achieving almost exactly the same results
- GPT-4 level models
- all struggling with hallucinations and boneheaded errors
- hardly any investment in anything else
Our interpretation
- The ordinary reading is a claim about the dominant direction of industry work, broadly comparable frontier-model capabilities, persistent errors, and a negligible share of investment in alternative approaches.
- The qualification 'for the most part' allows exceptions. An isolated alternative project cannot by itself contradict the investment claim.
- Use July 2024 through publication as the latest publicly observable industry state. This is a current research and product snapshot, rather than a claim about a particular financial quarter.
- Interpret 'investment' as resources devoted to AI development. Marcus supplies neither a spending definition nor an accounting basis.
- Do not invent numerical cutoffs for 'almost exactly' or 'hardly any'. A definitive assessment requires evidence strong enough to withstand ordinary variations in those terms.
Counts as supported if: Contemporaneous comparisons show broadly similar frontier capabilities and persistent factual or elementary reasoning errors, while comparable resource-allocation records show that major companies overwhelmingly concentrated AI development on scaling LLMs, with negligible investment elsewhere.
Counts as contradicted if: Comparable evidence shows substantial, systematic capability differences inconsistent with the claimed clustering, or a material industry-wide share of AI investment in alternative approaches. Exceptions already allowed by 'for the most part' are insufficient alone.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Meta's disclosures support the picture of large-scale LLM development, competitive frontier performance and continuing hallucination problems. Other disclosures show capability improvements and research beyond that pattern. Those examples do not settle a claim qualified by 'for the most part'. The central gap is investment allocation: the retrieved records do not establish how much major companies devoted to alternatives relative to LLM scaling. The claim also supplies no accounting basis. A definitive overall verdict would therefore exceed the evidence.
Limitations
- Company evaluations are not a substitute for consistent independent comparisons.
- The claim combines model performance, errors and investment allocation.
- Some contextual evidence describes June rather than the July target period.
Evidence (4 items)
Context only: Meta,
largest model
Why it matters here: Meta documents a larger model trained on extensive data. Its evaluations section also reports competitiveness with GPT-4, GPT-4o and Claude 3.5 Sonnet, but neither statement measures industry-wide investment allocation.
- Publisher
- Meta
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: Meta authors via arXiv,
Hallucinations remain
Why it matters here: The technical report acknowledges persistent hallucination problems. This supports that part of Marcus's description, but does not supply a common error measurement across every major lab or settle the investment component.
- Publisher
- Meta authors via arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Anthropic,
64%
Why it matters here: This is material counterevidence to a blanket impression of negligible capability differences. It is a June internal comparison with an earlier Anthropic model, rather than a July cross-industry evaluation.
- Publisher
- Anthropic
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: Meta,
symbols and audio
Why it matters here: Meta describes research beyond simply enlarging text LLMs. This qualifies the picture of uniform development, but the earlier announcement does not quantify its share of investment.
- Publisher
- Meta
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
Comparable contemporaneous model evaluations and resource-allocation records covering a defined set of major companies, with consistent treatment of scaling, alternative approaches and shared infrastructure.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
So for the most part, we’ve been stuck in a world in which all the major tech companies are building exactly the same thing (bigger and bigger LLMs based on more data), achieving almost exactly the same results (a passel of GPT-4 level models, hardly different from each other, all struggling with hallucinations and boneheaded errors), and hardly any investment in anything else.
- Where
- paragraph 51
- Published
- 2024-07-28
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-alphaproof-alphageometry-chatgpt-bb6581-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-alphaproof-alphageometry-chatgpt-bb6581-a1.