“What we are going to see, in the fullness of time, I promise you, is that Gemini is more or less in the same ball park as GPT-4, handy for a bunch of things, but untethered in reality, still with dicey, unpredictable reasoning, and a very limited understanding of the world.”
Our reading, the sentence that was checked:
Gemini will be shown to still have dicey, unpredictable reasoning and a very limited understanding of the world.
Assessment: Unresolved No deadline stated on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, The truth about Google’s amazing new demo, paragraph 10.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-the-truth-about-googles-amazing-new-67ab22-2-a1
We recorded 6 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-12-07. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “in the fullness of time”
- Deadline state when assessed
- No deadline stated
- Deadline state on 2026-10-29
- No deadline stated (the release’s reference date)
Conditions
- in the fullness of time (qualifier, stated by the speaker)
From the speaker
- in the fullness of time
- still with dicey, unpredictable reasoning
- a very limited understanding of the world
Our interpretation
- The target is the original Gemini generation announced in December 2023.
- Dicey, unpredictable reasoning means substantial failures or instability across reasoning problems and reasonable changes to their presentation.
- Limited understanding of the world means poor robust performance on practical commonsense and world-model tasks. It is not a claim about consciousness.
- The context makes novel reasoning important; success or failure on familiar benchmark questions alone is not decisive.
- In the fullness of time is an open-ended time expression, not a conditional antecedent.
Counts as supported if: Controlled evaluations establish substantial reasoning failures and instability, together with broad limitations on novel commonsense or world-model tasks.
Counts as contradicted if: Broad controlled evaluations establish robust reasoning and generalization on novel world-understanding tasks, sufficient to defeat the ordinary description of substantial and unpredictable limitations.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No deadline stated
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Independent evaluations show real reasoning weaknesses in Gemini Pro, including reduced robustness on complex questions. Another study found competitive commonsense capabilities as well as specific weaknesses. That supports a narrower claim about limitations, but does not decisively establish the full prediction about unpredictable reasoning and very limited world understanding across the original generation. The forecast has no deadline, and the retrieved evidence does not meet the early-resolution rule.
Limitations
- Competitive benchmark performance does not prove broad world understanding.
- Benchmark failures do not by themselves prove unpredictability or failure on genuinely novel tasks.
- The cited evaluations are preprints with limited model and dataset coverage.
Evidence (2 items)
Supports the claim: Akter et al., via arXiv,
longer, more complex questions
Why it matters here: This establishes a concrete reasoning limitation in Gemini Pro as question complexity increases. It does not by itself establish unpredictable outputs, novel-task generalization failure, or the same limitation in Ultra.
- Publisher
- Akter et al., via arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Wang and Zhao, via arXiv,
competitive commonsense reasoning capabilities
Why it matters here: The study found meaningful commonsense competence alongside temporal, social, and contextual weaknesses in Pro and Pro Vision. These positive findings limit a sweeping interpretation of very limited understanding, but do not establish robust generalization on novel tasks.
- Publisher
- Wang and Zhao, via arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Replicated evaluations of the original Gemini models on novel reasoning and world-understanding tasks, including equivalent prompt variations, positive controls, and checks for training-data overlap.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, The truth about Google’s amazing new demo
What we are going to see, in the fullness of time, I promise you, is that Gemini is more or less in the same ball park as GPT-4, handy for a bunch of things, but untethered in reality, still with dicey, unpredictable reasoning, and a very limited understanding of the world.
- Where
- paragraph 10
- Published
- 2023-12-07
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-the-truth-about-googles-amazing-new-67ab22-2-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-the-truth-about-googles-amazing-new-67ab22-2-a1.