“This is the uncanny valley we are in, and one that we occupy for a while.”
Our reading, the sentence that was checked:
AI systems such as video, image, and language generators are now in an uncanny valley of impressive output with basic errors, and will stay there for a while.
Assessment: Unresolved No deadline stated on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, DeepFakes versus Deep Understanding, paragraph 6.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-deepfakes-versus-deep-understanding-e025b6-a1
We recorded 4 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-10-02. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “for a while”
- Deadline state when assessed
- No deadline stated
- Deadline state on 2026-10-29
- No deadline stated (the release’s reference date)
Conditions
- generative video, image, and language systems as illustrated (scope, stated by the speaker)
From the speaker
- the uncanny valley we are in
- one that we occupy for a while
- A cat holding a remote control shouldn’t have a human hand
- for all the brushwork, the painting never changed
- the same thing over and over in language systems
Our interpretation
- The ordinary reading is that impressive generative outputs will continue to coexist with basic errors in physical behavior, object relationships, and everyday reasoning.
- The examples identify the relevant kinds of errors; they do not commit Marcus to every system failing every task or to progress stopping.
- “For a while” expresses persistence but supplies no deadline or minimum duration. I do not convert it into a fixed number of months or years.
- The references to video, image, and language systems describe scope, not an antecedent that must occur before the forecast applies.
- Later evaluations can support the predicted pattern without conclusively settling its unspecified duration.
Counts as supported if: Repeated evaluations after publication show that capable video, image, and language generators continue making basic semantic or commonsense errors. Conclusive support additionally requires a defensible duration for “for a while”; the source supplies none.
Counts as contradicted if: Broad, reproducible evaluations show that the described combination of impressive outputs and recurring basic errors ceased within the duration the forecast committed to. Improvement on selected tasks alone would not contradict the claim, and that duration is unspecified.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No deadline stated
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The evidence supports the predicted pattern: later evaluations found physical errors in generated videos, object-relation errors in images, and plausible explanations of basic reasoning mistakes. There was also substantial progress: a later human evaluation found the original image benchmark largely solved. Marcus did not say how long “for a while” meant or what would count as leaving this state. These studies therefore support persistence but do not supply a decisive event that settles the open-ended forecast under the stated rules.
Limitations
- This outcome does not mean the observed persistence is unsupported; it reflects the missing duration and resolution threshold.
- Benchmark failures do not establish that all systems fail, that progress stopped, or that systems lack understanding in every sense.
- Evidence dates identify the years of the reported evaluations; exact experiment dates are not supplied by the cited passages.
Evidence (4 items)
Supports the claim: arXiv,
far from accurately simulating
Why it matters here: The evaluation reports that its best tested model satisfied both the caption and physical laws in 39.6% of instances, despite substantial video-generation capabilities. This supports persistence of the described video failures, without establishing their frequency in ordinary use.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: arXiv,
spatial reasoning
Why it matters here: GenEval documents failures to arrange objects and assign their attributes correctly roughly a year after the post. The study also reports improvements, so it supports coexistence of capability and errors rather than stagnation.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: arXiv,
wrong solutions
Why it matters here: The revised study reports failures on simple family-relationship reasoning problems accompanied by plausible explanations. This matches the language-system pattern, but a narrow problem family does not establish failure rates across everyday language tasks.
- Where
- Abstract, version 5
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv,
Further, according to the human evaluation, GenEval is now saturated.
Why it matters here: Human assessment found that newer image models largely solved the original GenEval tasks and that its automated scoring had drifted from human judgment. This limits extrapolation from e2 to later systems, although the paper also reports room for improvement on its expanded benchmark.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
An independently documented clarification of the intended duration and exit criterion, followed by comparable evaluations across video, image, and language systems. Without that clarification or an unambiguous decisive event, additional failures strengthen the pattern but do not close the forecast.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, DeepFakes versus Deep Understanding
This is the uncanny valley we are in, and one that we occupy for a while.
- Where
- paragraph 6
- Published
- 2022-10-02
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-deepfakes-versus-deep-understanding-e025b6-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-deepfakes-versus-deep-understanding-e025b6-a1.