“The argument I made there is that … systems like DALL-E don’t “understand what wheels are—or what they are for”, because (as I discovered on Sunday) they systematically fail to properly illustrate requests like these:”
Our reading, the sentence that was checked:
Systems like DALL-E do not understand what wheels are or what they are for, because they systematically fail to properly illustrate certain requests he tested.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, The New Sport of Misrepresenting AI Criticism, paragraph 20.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-the-new-sport-of-misrepresenting-7841f0-a1
We recorded 8 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-09-23. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- systems like DALL-E (scope, stated by the speaker)
- as discovered on Sunday (qualifier, stated by the speaker)
From the speaker
- systems like DALL-E
- understand what wheels are—or what they are for
- as I discovered on Sunday
- systematically fail to properly illustrate requests like these
Our interpretation
- The plain reading is a repeatable failure to depict wheel parts and functions correctly, rather than an occasional malformed image.
- Sunday means September 18, 2022. The claim describes systems tested then and their capabilities when the post appeared.
- The linked essay identifies Craiyon and Stable Diffusion experiments. These must not be treated as experiments on OpenAI's DALL-E 2.
- Correct depiction is an observable test of the claimed understanding. Rendering failures alone do not establish the absence of every internal representation of wheels.
Counts as supported if: Independent, documented tests of the referenced contemporary systems and prompts show recurring failures to identify or depict wheels and their functions, with controls separating these failures from general problems following instructions or rendering labels.
Counts as contradicted if: Repeated tests of the same contemporary systems reliably satisfy the referenced wheel and function requests, showing that the alleged systematic failure does not hold.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Independent research documents related image-generation failures, but it does not reproduce the wheel tests Marcus referenced. The linked essay also concerns Craiyon and Stable Diffusion, so its examples cannot establish DALL-E 2's performance. The available evidence does not settle either the specific systematic-failure claim or the inference about understanding.
Limitations
- The independent study predates the target period and tests different relations.
- A failed drawing does not by itself identify the underlying cause.
Evidence (1 item)
Context only: arXiv; Colin Conwell and Tomer Ullman,
~22%
Why it matters here: The authors report systematic relational difficulties in DALL-E 2. These earlier tests do not reproduce the referenced wheel prompts or establish failures in the other named systems.
- Where
- Abstract
- Publisher
- arXiv; Colin Conwell and Tomer Ullman
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Independent archived tests or a documented replication using the September 2022 model versions, the referenced prompts, complete output samples, and controls for instruction-following failures.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, The New Sport of Misrepresenting AI Criticism
The argument I made there is that … systems like DALL-E don’t “understand what wheels are—or what they are for”, because (as I discovered on Sunday) they systematically fail to properly illustrate requests like these:
- Where
- paragraph 20
- Published
- 2022-09-23
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-the-new-sport-of-misrepresenting-7841f0-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-the-new-sport-of-misrepresenting-7841f0-a1.