“Although GPT-4 is clearly better than GPT-3 and 3.5, it is not so far as I can tell, qualitatively better, only quantitatively better; as noted above, the limits remain more or less the same.”
Our reading, the sentence that was checked:
So far as Marcus can tell, GPT-4, though clearly better than GPT-3 and 3.5, is only quantitatively and not qualitatively better, and its limits remain more or less the same.
Assessment: Can’t be checked Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, GPT-4’s successes, and GPT-4’s failures, paragraph 6.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-gpt-4s-successes-and-gpt-4s-failures-7cd602-a1
We recorded 12 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-03-15. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- so far as I can tell (qualifier, stated by the speaker)
From the speaker
- clearly better than GPT-3 and 3.5
- not so far as I can tell, qualitatively better
- only quantitatively better
- the limits remain more or less the same
Our interpretation
- The statement combines a checkable observation about retained limitations with a qualified judgment about the kind of improvement.
- No ordinary, unambiguous test distinguishes quantitative from qualitative improvement across the capabilities discussed.
- Treating any surviving limitation as proof that no qualitative improvement occurred would strengthen the statement beyond what the evidence can establish.
- Marcus acknowledges vision in the supplied context, so image input alone should not be treated as an automatic contradiction.
This claim is not graded on evidence. The record checks that the words are quoted correctly and attributed to the right person, and that the claim is classified correctly.
Current assessment
Not graded on evidence
Outcome: Can’t be checked
The statement is a recommendation, a value judgment, or too open to test. It is recorded, not scored.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The persistence of specific limitations is checkable, but it cannot by itself settle whether improvement was qualitative. Marcus qualifies that judgment and gives no test separating the categories. The visual application is relevant contrary context, yet he already acknowledges vision. Scoring the complete statement would require imposing a disputed definition.
Limitations
- This does not reject the checkability of narrower claims about particular limitations or benchmark improvements.
- The visual-assistant announcement is not a matched comparative evaluation.
Evidence (1 item)
No outcome evidence: a claim of this kind is recorded and attributed, not scored.
What would settle it
A definition of qualitative improvement tied to specified capabilities and comparison tests would make a narrower claim adjudicable.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, GPT-4’s successes, and GPT-4’s failures
Although GPT-4 is clearly better than GPT-3 and 3.5, it is not so far as I can tell, qualitatively better, only quantitatively better; as noted above, the limits remain more or less the same.
- Where
- paragraph 6
- Published
- 2023-03-15
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-gpt-4s-successes-and-gpt-4s-failures-7cd602-a1: Current. Outcome: Can’t be checked. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-gpt-4s-successes-and-gpt-4s-failures-7cd602-a1.