“Dubious Google Gemini videos aside, nobody is close to passing that yet.”
Our reading, the sentence that was checked:
As of December 2023, no one, Google's Gemini included, was close to passing the Comprehension Challenge.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Nope, the Turing Test has not been solved, paragraph 11.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-nope-the-turing-test-has-not-been-9086b0-a1
We recorded 4 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-12-09. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- setting aside dubious Google Gemini demo videos (qualifier, stated by the speaker)
From the speaker
- Dubious Google Gemini videos aside
- nobody is close
- to passing that yet
Our interpretation
- That refers to the Comprehension Challenge, not the Turing Test.
- The ordinary reading of close is reliable performance across most of the challenge's breadth, with limited remaining gaps relative to a competent human viewer.
- No numerical proximity threshold is supplied, so none is invented.
- The Gemini qualification discounts promotional demonstrations; it does not exclude Gemini from the capability claim or make the claim conditional.
- For this ongoing capability claim, use December 2023 through December 9 as the latest reporting window.
Counts as supported if: Broad, relevant evaluations show substantial remaining gaps across the challenge's central requirements for all leading contemporary systems, including Gemini.
Counts as contradicted if: A system existing by December 9, 2023 shows reliable, broadly human-comparable open-ended video comprehension across most challenge requirements, with only limited remaining gaps.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Contemporary research showed substantial progress alongside limitations. The reviewed results do not measure distance from passing Marcus's challenge, and he supplied no threshold for close. It would be unfair either to equate strong narrower benchmarks with near-passing performance or to infer that nobody was close from the absence of a full passing demonstration.
Limitations
- The result depends materially on the unspecified meaning of close.
- The benchmarks cover narrower tasks than the challenge.
- The archived Gemini report was published after the post and is used only as context.
Evidence (3 items)
Context only: arXiv; Enxin Song and colleagues,
MovieChat’s performance is hindered by the pretrained short video understanding model.
Why it matters here: MovieChat's reported long-video progress and remaining limitations are both relevant. Neither establishes how close it was to a broad human-level comprehension standard.
- Publisher
- arXiv; Enxin Song and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv; Kunchang Li and colleagues,
we randomly sample 10 multiple-choice QAs for each task.
Why it matters here: The paper reports meaningful temporal-understanding performance, including for GPT-4V. Its sampled multiple-choice comparison does not measure near-passing performance on arbitrary open-ended video questions.
- Publisher
- arXiv; Kunchang Li and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv; Google Gemini Team,
we sample 16 equally-spaced frames from each video clip
Why it matters here: The Gemini 1.0 report describes strong results on selected video benchmarks using sampled frames. These results supply relevant counterweight to a broad dismissal, but do not establish proximity to Marcus's full challenge or a human-viewer standard.
- Publisher
- arXiv; Google Gemini Team
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
A defensible proximity criterion and a broad evaluation of December 2023 systems against human viewers, covering open-ended explanations, motives, humor, and unfamiliar video content.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Nope, the Turing Test has not been solved
Dubious Google Gemini videos aside, nobody is close to passing that yet.
- Where
- paragraph 11
- Published
- 2023-12-09
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-nope-the-turing-test-has-not-been-9086b0-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-nope-the-turing-test-has-not-been-9086b0-a1.