“I am willing to bet SlateStarCodex now (terms to be negotiated) that if OpenAI gives us unrestricted access to GPT-4, whenever that is released, and assuming that is basically the same architecture but with more data, that within a day of playing around with it, Ernie and I will still be able find lots of examples of failures in physical reasoning, temporal reasoning, causal reasoning, and so forth.”
Our reading, the sentence that was checked:
If given unrestricted access to GPT-4 and it is basically the same architecture with more data, Marcus and Davis will find many failures in physical, temporal, and causal reasoning within a day.
Assessment: Unresolved No deadline stated on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, What does it mean when an AI fails? A Reply to SlateStarCodex’s riff on Gary Marcus, paragraph 48.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-what-does-it-mean-when-an-ai-fails-e02fad-a2
We recorded 14 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-06-07. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Conditional forecast
- Deadline, as the speaker wrote it
- “within a day of playing around with it”
- Deadline state when assessed
- No deadline stated
- Deadline state on 2026-10-29
- No deadline stated (the release’s reference date)
Conditions
- OpenAI gives unrestricted access to GPT-4 (antecedent, stated by the speaker)
- GPT-4 is basically the same architecture with more data (antecedent, stated by the speaker)
From the speaker
- if OpenAI gives us unrestricted access to GPT-4
- whenever that is released
- assuming that is basically the same architecture but with more data
- within a day of playing around with it
- Ernie and I
- lots of examples
- physical reasoning, temporal reasoning, causal reasoning
Our interpretation
- Both access and architecture conditions must be established before scoring the forecast.
- The named testers are Marcus and Ernest Davis. Other researchers' results cannot substitute for their first day of testing.
- The deadline is relative to qualifying access and the start of testing; the post supplies no calendar deadline.
- Lots means numerous distinct failures across the stated kinds of reasoning, but no exact count was negotiated.
- Unrestricted access is not assumed to mean ordinary access to a rate-limited public service.
Counts as supported if: Evidence establishes both antecedents and a timestamped first-day testing record shows Marcus and Davis finding numerous distinct failures across the specified reasoning areas.
Counts as contradicted if: Evidence establishes both antecedents and a sufficiently complete record of their first day shows that they did not find numerous qualifying failures.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No deadline stated
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
OpenAI's March 14, 2023 launch disclosure describes capped ChatGPT Plus access and a staged API rollout with default rate limits [e1]. It does not establish unrestricted access for Marcus and Davis, verify the same-architecture-with-more-data condition, or document their first day of testing [e1]. The supplied evidence therefore establishes neither the conditions needed to start the one-day window nor a qualifying testing result; the forecast remains unresolved with an open horizon [e1].
Limitations
- The public rollout disclosure does not establish whether Marcus and Davis separately received qualifying access [e1].
- The supplied record does not verify the architecture and additional-data condition or provide timestamped first-day testing logs.
- The proposed bet leaves 'lots,' 'unrestricted access' and the precise meaning of a day undefined.
Evidence (1 item)
Context only: OpenAI,
start inviting some developers today
Why it matters here: The launch describes a staged public rollout. It supplies no confirmation that Marcus and Davis received access satisfying the forecast's unrestricted-access condition.
- Where
- API section
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
Documentation establishing both antecedents, followed by independently checkable logs of Marcus and Davis's first qualifying day, including start time, model version, prompts, outputs and distinct failures across the specified reasoning areas. Applying the result also requires clear interpretations of 'lots,' 'unrestricted access' and the one-day window.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, What does it mean when an AI fails? A Reply to SlateStarCodex’s riff on Gary Marcus
I am willing to bet SlateStarCodex now (terms to be negotiated) that if OpenAI gives us unrestricted access to GPT-4, whenever that is released, and assuming that is basically the same architecture but with more data, that within a day of playing around with it, Ernie and I will still be able find lots of examples of failures in physical reasoning, temporal reasoning, causal reasoning, and so forth.
- Where
- paragraph 48
- Published
- 2022-06-07
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-what-does-it-mean-when-an-ai-fails-e02fad-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-what-does-it-mean-when-an-ai-fails-e02fad-a2.