“The authors wisely don’t want to put all their eggs in the basket of pretraining scaling (which used to fuel most people’s AGI fantasies), because by now they know perfectly well that pretraining scaling has hit a rough patch.”
Our reading, the sentence that was checked:
Pretraining scaling has hit a rough patch.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, The “AI 2027” Scenario: How realistic is it?, paragraph 40.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-the-ai-2027-scenario-how-realistic-4764ac-a2
We recorded 40 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-05-22. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- by now
- pretraining scaling has hit a rough patch
Our interpretation
- The ordinary reading is that increasing pretraining scale had encountered meaningful obstacles or disappointing capability gains. It does not mean that scaling had stopped working.
- I assess the current 2025 model-development cycle through the post date. Research releases have no standard quarterly reporting period.
- I assess the technical proposition, not the assertion about what the AI 2027 authors privately knew.
Counts as supported if: Contemporaneous development records or comparable scaling experiments document substantial obstacles or disappointing gains attributable to increasing pretraining resources.
Counts as contradicted if: Comparable records establish that the alleged recent difficulties did not occur. Evidence of continued improvement alone would not contradict a rough patch.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
On February 27, 2025, METR reported that GPT-4.5 performed between GPT-4o and o1 on its preliminary evaluations [e1]. OpenAI's release report that day attributed capability improvements to scaling the GPT paradigm [e2]. These reports do not isolate capability returns to increased pretraining resources or establish whether results fell below development expectations [e1, e2]. The January 2025 International AI Safety Report described data shortages as a plausible future bottleneck, not an established early-2025 constraint [e3]. Together, these records neither establish nor rule out a pretraining scaling rough patch during January 1–May 22, 2025 [e1, e2, e3].
Limitations
- No comparable resource-to-capability series separates pretraining scale from architecture, data, post-training, and inference-time computation.
- The evidence does not establish the extent of any difficulty across frontier laboratories.
- The claim supplies no severity threshold or comparison baseline for a rough patch.
Evidence (3 items)
Context only: METR,
between GPT-4o and o1.
Why it matters here: The measured result helps explain disappointment about GPT-4.5. It does not isolate pretraining returns: the comparison includes models with different training and reasoning approaches.
- Publisher
- METR
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: OpenAI,
capability improvements
Why it matters here: OpenAI reported improvements from scaling, including better benchmark results than GPT-4o. This weighs against stagnation, but improvements can coexist with a rough patch.
- Where
- Safety section
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: International AI Safety Report,
plausible bottleneck
Why it matters here: The report identifies a possible constraint, while discussing substantial uncertainty about when it would bind. A prospective bottleneck does not establish that pretraining had already encountered the claimed difficulty.
- Publisher
- International AI Safety Report
- Kind of source
- Secondary summary
- Describes events of
- Public from
What would settle it
Contemporaneous development records or controlled comparisons showing whether increased pretraining resources produced substantial obstacles or disappointing gains during January 1–May 22, 2025, with resource budgets, expected results, and other training changes disclosed.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, The “AI 2027” Scenario: How realistic is it?
The authors wisely don’t want to put all their eggs in the basket of pretraining scaling (which used to fuel most people’s AGI fantasies), because by now they know perfectly well that pretraining scaling has hit a rough patch.
- Where
- paragraph 40
- Published
- 2025-05-22
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-the-ai-2027-scenario-how-realistic-4764ac-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-the-ai-2027-scenario-how-realistic-4764ac-a2.