“The challenge of parts and wholes in generative images that Ernest Davis and I discussed here in December fared no better.”
Our reading, the sentence that was checked:
GPT-5 shows no improvement on the parts-and-wholes problem in generated images.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it., paragraph 23.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-gpt-5-overdue-overhyped-and-underwhelming-876436-a2
We recorded 23 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-08-09. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- challenge of parts and wholes in generative images
- that Ernest Davis and I discussed here in December
- fared no better
Our interpretation
- Read as: image generation in the GPT-5 ChatGPT product, at launch in August 2025, did no better on the parts-and-wholes prompts than the systems discussed in December 2024.
- Target period is August 2025.
Counts as supported if: Tests of GPT-5-era ChatGPT image generation on the Davis/Marcus parts-and-wholes prompts show failure rates no better than in the December 2024 tests.
Counts as contradicted if: Tests of GPT-5-era ChatGPT image generation on those prompts show clearly fewer failures than in the December 2024 tests.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Marcus said GPT-5 did no better on the parts-and-wholes image problems he and Ernest Davis had discussed in December. The packet contains no independent test of these prompts on GPT-5-era ChatGPT in August 2025. The closest evidence is Davis's own rerun, dated 8 July 2025, a month before GPT-5's launch (e1). It found ChatGPT-4o's image generation much improved since November 2024 on these prompts (e1). Failures in three tries fell from 11 to 4 of those examples, plus one failure on a new prompt (e2). That test predates GPT-5, and the packet does not confirm which image model GPT-5-era ChatGPT used. So the evidence neither supports nor contradicts the claim for August 2025.
Limitations
- The only systematic test predates GPT-5 and used ChatGPT-4o.
- Which image model GPT-5 ChatGPT used at launch is unconfirmed.
- The reading of 'fared no better' (no improvement versus still failing) changes the result.
Evidence (2 items)
Context only: Ernest Davis, NYU,
The bottom line is that, on the examples in this experiment, ChatGPT is very substantially improved over November 2024, but still has room for improvement.
Why it matters here: One month before GPT-5, Davis found ChatGPT's image generation much better than in November 2024 on these parts prompts, though not perfect. It predates GPT-5, so it is context, not outcome evidence.
- Where
- section Overall
- Publisher
- Ernest Davis, NYU
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: Ernest Davis, NYU,
In November 2024, There were 11 examples where ChatGPT could not produce a correct image in three tries; in July 2025, GPT-4o failed on only four of those plus the new problem.
Why it matters here: Failures dropped from 11 to 4 (plus one new one) between November 2024 and July 2025. If GPT-5 used the same image engine, this would cut against 'fared no better', but that link is not shown.
- Where
- section Overall
- Publisher
- Ernest Davis, NYU
- Kind of source
- Independent measurement
- Describes events of
- Public from
What would settle it
A dated, independent rerun in August 2025 of the Davis parts-and-wholes prompts on GPT-5-era ChatGPT image generation, compared with the late-2024 results.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
The challenge of parts and wholes in generative images that Ernest Davis and I discussed here in December fared no better.
- Where
- paragraph 23
- Published
- 2025-08-09
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-gpt-5-overdue-overhyped-and-underwhelming-876436-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-gpt-5-overdue-overhyped-and-underwhelming-876436-a2.