“Without external aids it won’t be able beat Meta’s Cicero in Diplomacy; it won’t be able to drive a car reliably; it won’t be able to reliably guide a robot like Optimus to be anything like as versatile as Rosie the Robot.”
Our reading, the sentence that was checked:
Without external aids, GPT-4 won't be able to beat Meta's Cicero in Diplomacy, won't be able to drive a car reliably, and won't be able to reliably guide a robot like Optimus to be anything like as versatile as Rosie the Robot.
Assessment: Unresolved No deadline stated on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, What to Expect When You’re Expecting … GPT-4, prediction 5 of 7.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-what-to-expect-when-youre-expecting-177f3a-a1
We recorded 27 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-12-25. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No deadline stated
- Deadline state on 2026-10-29
- No deadline stated (the release’s reference date)
Conditions
- without external aids (qualifier, stated by the speaker)
From the speaker
- Without external aids
- won’t be able beat Meta’s Cicero in Diplomacy
- won’t be able to drive a car reliably
- won’t be able to reliably guide a robot like Optimus
- anything like as versatile as Rosie the Robot
Our interpretation
- The claim contains three capability limitations, each requiring its own evidence.
- Minimal interfaces for receiving observations and issuing actions are allowed. Separate planning, retrieval, perception or autonomous-control systems count as external aids.
- Beat Cicero ordinarily means demonstrated competitive superiority, rather than one lucky result.
- Reliable driving requires sustained safe performance across realistic driving conditions.
- Rosie is read as broad household-task versatility, rather than a literal requirement to reproduce a fictional character.
- Without external aids is a configuration restriction, not an external triggering event.
Counts as supported if: Controlled evaluations establish all three limitations for GPT-4 under the specified restriction.
Counts as contradicted if: A reproducible evaluation establishes at least one asserted capability under the restriction. If another limitation is independently established, the combined claim is mixed.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No deadline stated
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The available studies are informative but do not settle the conjunction. Richelieu reports weak basic-model play and improved performance with added modules. Talk2Drive uses a separate driving controller. Alter3 demonstrates humanoid motion, not Rosie-like household versatility. These findings establish neither all three limitations nor a qualifying counterexample. The absence of a matching demonstration is not proof of inability.
Limitations
- The assistance boundary materially affects which demonstrations qualify.
- Driving reliability and household versatility lack stated thresholds.
- Experiment dates are reported only at year precision where exact dates were unavailable.
Evidence (3 items)
Context only: Guan et al., arXiv,
As can be seen, direct use of vanilla LLM yields relatively poor results.
Why it matters here: The ablation points toward weak unaided play, while the augmented Richelieu system competes successfully with Cicero. Neither result establishes the complete three-part claim, and the augmented result fails the no-external-aids restriction.
- Publisher
- Guan et al., arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Cui et al., arXiv,
Specifically, the LMPs adjust control parameters like the look-ahead distance and look-ahead ratio to optimize pure pursuit [24] performance.
Why it matters here: Talk2Drive uses GPT-4 to configure a separate driving system, including an Autoware controller. Its vehicle results therefore do not demonstrate reliable driving by GPT-4 alone.
- Publisher
- Cui et al., arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Yoshida, Masumori and Ikegami, arXiv,
We report the development of Alter3, a humanoid robot capable of generating spontaneous motion using a Large Language Model (LLM), specifically GPT-4.
Why it matters here: This is relevant evidence that GPT-4 can contribute to humanoid motion. The reported poses and action sequences do not test reliable, broadly versatile household work.
- Publisher
- Yoshida, Masumori and Ikegami, arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Controlled tests of the three specified capabilities, with model versions, assistance, task coverage and reliability disclosed. A qualifying success could refute one component; support for the whole claim requires all three limitations.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, What to Expect When You’re Expecting … GPT-4
Without external aids it won’t be able beat Meta’s Cicero in Diplomacy; it won’t be able to drive a car reliably; it won’t be able to reliably guide a robot like Optimus to be anything like as versatile as Rosie the Robot.
- Where
- prediction 5 of 7
- Published
- 2022-12-25
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-what-to-expect-when-youre-expecting-177f3a-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-what-to-expect-when-youre-expecting-177f3a-a1.