“It’s not like other systems are faring much better.”
Our reading, the sentence that was checked:
Other generative AI systems such as Grok are not faring much better than GPT-5.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it., paragraph 52.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-gpt-5-overdue-overhyped-and-underwhelming-efa4ab-a2
We recorded 23 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-08-09. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- other systems
- not ... faring much better
- the field of generative AI as a whole
- Grok gets literally everything wrong
Our interpretation
- The adjacent discussion of inaccurate scientific summaries makes factual accuracy and reliability the plain reading of 'faring', rather than revenue or popularity.
- The comparison concerns GPT-5 and competing systems available at its launch, through the post date.
- A materially better competitor would show a substantial, consistent reliability advantage across representative comparable tasks. I do not invent a numerical margin for 'much'.
Counts as supported if: Comparable evaluations of launch-period systems show that leading competitors generally have similar or worse factual reliability than GPT-5, without a consistent substantial advantage.
Counts as contradicted if: Comparable evaluations show leading competitors consistently and substantially outperforming GPT-5 on representative factual-reliability tasks.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Vectara's November 2025 evaluation, published November 19, reported hallucination rates above 10% for GPT-5, Grok-4 and several other models on its new document-summarization benchmark [e1]. This later, task-specific comparison does not establish whether competitors had a substantial, consistent factual-reliability advantage during August 7–9, 2025 [e1]. It therefore does not settle the launch-period claim [e1].
Limitations
- The packet contains no verified primary comparison from the August 7–9 launch window.
- The November evaluation uses a changed dataset and covers summarization rather than representative factual reliability across uses.
- Shared hallucination rates above a threshold do not establish similar performance or the absence of a substantial competitor advantage.
Evidence (1 item)
Context only: Vectara,
GPT-5, GPT-OSS-120B, Grok-4, or Deepseek-R1 all have a hallucination rate > 10%.
Why it matters here: The benchmark finds hallucinations across several model families on document summarization. It uses a later evaluation and changed dataset, so it cannot settle their relative reliability during GPT-5's launch.
- Publisher
- Vectara
- Kind of source
- Benchmark result
- Describes events of
- Public from
What would settle it
Dated evaluations of launch-period GPT-5 and leading competitors across representative factual tasks, matching prompts, tools, reasoning settings and scoring, with uncertainty and abstention reported sufficiently to judge the size and consistency of any reliability advantage.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
It’s not like other systems are faring much better.
- Where
- paragraph 52
- Published
- 2025-08-09
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-gpt-5-overdue-overhyped-and-underwhelming-efa4ab-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-gpt-5-overdue-overhyped-and-underwhelming-efa4ab-a2.