“They fail to “understand” simple concepts, such as “design to used with only one hand.””
Our reading:
GenAI systems fail to understand simple concepts such as designing for one-handed use.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The claim has a fair behavioral reading: contemporary generators repeatedly missed simple design requirements, while sometimes succeeding. The retrieved demonstration reports both failures and successes on unusual objects. The design benchmark uses expanded prompts and does not establish functional compliance. Neither settles the controller example or the broader tendency. Marcus's own description cannot supply the missing independent evidence.
Caveats
- The original controller demonstration was not independently verified.
- Informal examples cannot establish a general failure rate.
- Observable compliance does not settle philosophical claims about understanding.
Evidence (2 items)
Context only: NLeseul, Thinking About Stuff,
The “spherical table” problem
Why it matters: This informal demonstration reports a clear failure alongside better results on other unusual objects. It neither measures the frequency of functional-design failures across contemporary systems nor tests one-handed controllers.
- Publisher
- NLeseul, Thinking About Stuff
- Kind of source
- Demonstration
- About events on
- Published
Context only: Kevin Lin and colleagues, arXiv,
the expanded text prompt
Why it matters: DEsignBench provides a contemporary comparison of image generators in design scenarios. Expanded prompts and visual-design evaluation do not directly settle compliance with an ordinary request for one-handed operation.
- Publisher
- Kevin Lin and colleagues, arXiv
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- Read in context, this concerns contemporary image generators repeatedly failing to translate simple requirements into suitable designs. It does not say every output fails.
- Treat understanding operationally: whether generated designs satisfy the requested requirement. This cannot establish the presence or absence of an internal mental state.
- Ordinary prompts may state the functional requirement without specifying the complete solution. Occasional success, especially with detailed instructions, is compatible with the claim.
- The target is capability as of February 8, 2024. Earlier tests of the contemporary systems are eligible. No common starting date is supplied, so the lower date boundary is left unspecified rather than imposing a one-week window.
Right if: Documented, reproducible tests of contemporary image generators show recurring failures to satisfy simple functional design requirements, including one-handed operation or closely comparable constraints, with ordinary prompts and disclosed outputs.
Wrong if: Comparable tests show that contemporary image generators reliably satisfy those requirements across ordinary prompt variations and repeated outputs. A selected successful image alone is insufficient.
We wrote this test after the outcome was known.
What would settle it
An archived evaluation of the contemporary systems with model versions, ordinary prompts, complete output samples and explicit functional assessments, including one-handed controller use or closely comparable requirements.
We found 6 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Further Trouble in Hinton City, paragraph 10 (item 3 of 3).
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-29. The post may have been edited after 2024-02-08.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Further Trouble in Hinton City
They fail to “understand” simple concepts, such as “design to used with only one hand.”
- Published
- 2024-02-08
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.