“I was right on the money then.”
Our reading, the sentence that was checked:
Marcus's December 2022 predictions about GPT-4 were accurate.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, What to Expect When You’re Expecting … GPT-5, paragraph 14.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-what-to-expect-when-youre-expecting-62e-6f365c-a2
We recorded 15 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-07-27. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- I was right on the money then.
- an earlier essay that I posted on Christmas Day 2022
- If all seven predictions prove correct
- Within a decade, maybe much less
- In retrospectives written in 2043
Our interpretation
- Read as a retrospective claim that the seven GPT-4 predictions were substantively accurate. This does not require treating rhetorical descriptions as literal technical claims.
- The assessment period starts in 2023, the year the original essay explicitly anticipated, and ends on 2025-07-27, when Marcus asserted vindication. Later developments cannot establish what had already happened by that assertion.
- Persistent unreliability means material failures in relevant tasks, not an absence of improvement or usefulness. Successful integrations alone do not establish dependable interpretation of user intentions.
- The seventh prediction includes longer-term claims about future architectures, a shift within a decade of December 2022, and historical judgments in 2043. These remain part of the referenced list; an unelapsed prediction is not a failed prediction.
- A narrower reading referring only to the near-term warnings about errors and safety would receive substantially stronger support.
Counts as supported if: Contemporaneous evaluations substantiate the material predictions across the referenced list: persistent errors, unreliable reasoning and medical advice, hallucinations and guardrail weaknesses, unreliable downstream use, limited generality, unresolved alignment, and the claimed architectural trajectory. Supporting several warnings alone does not settle the whole list.
Counts as contradicted if: Comparable evidence demonstrates that material predicted limitations did not persist or that a substantive prediction was false. A mixed finding requires both substantiated and refuted parts. Improvements, successful demonstrations, and unelapsed deadlines do not alone constitute refutation.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
OpenAI acknowledged GPT-4 hallucinations and reasoning errors on March 14, 2023 [e2]. Studies published on June 20 and July 16, 2023 documented bias under targeted prompts and errors in clinical assistance, respectively [e3, e4]. These findings substantiate those specific warnings [e2, e3, e4]. OpenAI's June 13, 2023 function-calling announcement described improved integration, but did not establish dependable interpretation of user intentions [e5]. The original December 2022 list also predicted a shift within a decade, future AGI architecture and historical judgments in 2043; these cannot be treated as established outcomes by July 27, 2025 [e1]. The cited findings therefore do not settle the accuracy of the entire seven-prediction list [e1–e5].
Limitations
- The assessment reads the retrospective as referring to all seven predictions; its intended scope is not explicit.
- The packet does not settle the specific unaided Diplomacy, driving and robotics comparisons or downstream reliability.
- The independent studies were preprints with bounded tasks; their cited passages do not give exact testing dates.
- Pending longer-term predictions and integration improvements do not establish a refuted component.
- The assessment concerns outcomes through July 27, 2025; later fulfillment cannot establish prior accuracy.
Evidence (5 items)
Supports the claim: OpenAI,
it still is not fully reliable
Why it matters here: Supports the central predictions that GPT-4 would retain reasoning errors and hallucinations. The same section reports improvements, so persistent limitations should not be confused with a lack of progress.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: Wang et al., arXiv,
the output bias of GPT models increases significantly.
Why it matters here: The study evaluates GPT-4 and GPT-3.5 and finds that misleading prompts can elicit biased outputs, supporting the guardrail and alignment warnings. It also reports substantial resistance to biased statements under benign prompts; the failures are not representative of every interaction.
- Publisher
- Wang et al., arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: Zhang et al., arXiv,
overlooking crucial medical findings
Why it matters here: Supports the original prediction that medical answers could contain consequential mistakes despite strong performance. This is a bounded clinical-assistant evaluation, not proof that every medical application is unreliable.
- Publisher
- Zhang et al., arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Gary Marcus, Marcus on AI,
If all seven predictions prove correct
Why it matters here: Establishes the referenced set of seven predictions, not their accuracy. The preceding prediction includes a decade-long shift, future AGI architecture, and retrospectives in 2043.
- Publisher
- Gary Marcus, Marcus on AI
- Kind of source
- Secondary summary
- Describes events of
- Public from
Context only: OpenAI,
more reliably connect GPT's capabilities
Why it matters here: The GPT-4 update enabled structured function calls, including database and assistant uses, qualifying any blanket suggestion that integration was impractical. It does not establish dependable interpretation of intentions or error-free downstream behavior; the announcement also acknowledges unintended-action risks.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
Clarification of which predictions the retrospective covers, plus dated evidence concerning outcomes through July 27, 2025 that establishes or refutes the remaining material assertions. Eventual outcomes of the longer-term forecasts would require separate assessment.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, What to Expect When You’re Expecting … GPT-5
I was right on the money then.
- Where
- paragraph 14
- Published
- 2025-07-27
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-what-to-expect-when-youre-expecting-62e-6f365c-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-what-to-expect-when-youre-expecting-62e-6f365c-a2.