“No model could have done this in 2022. The graphics are vastly better than they used to be.”
Our reading, the sentence that was checked:
Image generation graphics quality has improved vastly since 2022, to a level no 2022 model could reach.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Spinal Tap meets John Malkovich, paragraph 15.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-spinal-tap-meets-john-malkovich-0500bc-a2
We recorded 4 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-08-31. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- compared to 2022 (qualifier, stated by the speaker)
From the speaker
- No model could have done this in 2022
- graphics are vastly better than they used to be
- 'this' = a sharp image of a unicyclist with a spoked wheel and pedals, made by ChatGPT-5 Thinking
Our interpretation
- Plain reading: AI image generation in mid-2025 is much better than in 2022. The 2022 tools generally could not produce a clean, sharp picture of a person on a correctly built wheel with spokes and pedals.
- The absolute 'no model could' is read as ordinary emphasis that this was beyond what 2022 tools reliably did. It is not read as a literal claim that no single 2022 sample could ever have looked like this.
- The period runs from 2022 to the post date, because the claim compares the two.
Counts as supported if: Sources showing that 2022 image tools mostly produced mangled or unusable bicycle and rider images, and that 2025 systems score far higher on measures of image quality or prompt accuracy.
Counts as contradicted if: Documented 2022 outputs or benchmarks showing that 2022 models routinely produced sharp, mechanically correct images of riders on cycles, or showing no large quality gap between 2022 and 2025 models.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The contract reads the claim plainly: AI image generation in mid-2025 is much better than in 2022, and a sharp rider on a correctly built wheel with spokes and pedals was beyond what 2022 tools reliably produced. It does not read 'no model could' as a literal claim about every possible 2022 output. In August 2022, CyclingTips tested Stable Diffusion, Midjourney, DreamStudio, Craiyon and Enstil on cycling prompts (e1). It judged most of the resulting images 'a pretty decisive no' for professional use. It also found some impressive artistic results, mainly from Midjourney (e1). On the GenEval benchmark, which checks whether images match the objects, counts and positions in a prompt, Stable Diffusion v1.5, a 2022 model, scores 0.43 overall (e2). A 2025 study reports that GPT-4o's image generator scores 0.84 on the same benchmark, about double (e3). Together, these support a large improvement from 2022 to 2025. They do not show that no 2022 model could ever produce such an image.
Limitations
- The evidence does not show that no 2022 model could produce a sharp, mechanically correct image of a rider. The 2022 test found some good outputs.
- GenEval measures whether images match the prompt, not sharpness or 'graphics' as such.
- DALL-E 2, which the 2022 test called the gold standard, was not part of that test.
- The ChatGPT-5 Thinking image described in the post was not independently inspected. The 2025 benchmark score is for GPT-4o's image generator.
Evidence (3 items)
Supports the claim: Velo / CyclingTips,
For most of the images above, the answer’s a pretty decisive no (as entertaining as they are to look at).
Why it matters here: In August 2022, CyclingTips tested Stable Diffusion, Midjourney, DreamStudio, Craiyon and Enstil on cycling prompts. It judged most of the realistic cycling images unusable, which fits the claim that 2022 tools could not reliably draw riders on correct cycles.
- Publisher
- Velo / CyclingTips
- Kind of source
- News report
- Describes events of
- Public from
Supports the claim: GenEval (Ghosh et al.),
Stable Diffusion v1.5 0.43
Why it matters here: Stable Diffusion v1.5 was released in 2022. It scores 0.43 overall on GenEval, a benchmark that checks whether images correctly show the objects, counts and positions the prompt asks for.
- Publisher
- GenEval (Ghosh et al.)
- Kind of source
- Benchmark result
- Describes events of
- Public from
Supports the claim: arXiv (Yan et al., GPT-ImgEval),
According to the table, GPT4o achieves the highest overall score of 0.84
Why it matters here: In 2025, GPT-4o's image generator (the one used in ChatGPT) scored 0.84 on the same GenEval benchmark. That is roughly double the 2022 model's 0.43, a large measured improvement.
- Publisher
- arXiv (Yan et al., GPT-ImgEval)
- Kind of source
- Preprint, not peer reviewed
- Arithmetic
- 0.84 / 0.43 (inputs: e2: Stable Diffusion v1.5 GenEval overall 0.43; e3: GPT-4o GenEval overall 0.84; about 1.95x, i.e. roughly double)
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Spinal Tap meets John Malkovich
No model could have done this in 2022. The graphics are vastly better than they used to be.
- Where
- paragraph 15
- Published
- 2025-08-31
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-spinal-tap-meets-john-malkovich-0500bc-a2: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-spinal-tap-meets-john-malkovich-0500bc-a2.