“my perspective that scaling was no longer working as effectively as it once did”
Our reading, the sentence that was checked:
Scaling of AI models is no longer working as effectively as it once did.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, OpenAI’s o3 and Tyler Cowen’s Misguided AGI Fantasy, paragraph 30.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-openais-o3-and-tyler-cowens-misguided-b9c3c2-a2
We recorded 4 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-04-17. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- scaling
- was no longer working as effectively as it once did
- in February
Our interpretation
- The plain reading is that increasing frontier LLM pretraining scale was producing smaller capability gains relative to the increase in resources than earlier scale-ups.
- The footnote recalls a view expressed in February 2025, so the target period is that month. The date within February and the earlier comparison period are unspecified.
- Comparing gains for similar proportional increases in training compute is an evaluator-selected measure of effectiveness, not a metric Marcus supplied.
- Pretraining is the working interpretation of scaling. Changes in inference-time compute, post-training, architecture, and data quality must be separated where possible.
Counts as supported if: Comparable measurements show that frontier pretraining scale-ups relevant to February 2025 yielded smaller capability gains for similar proportional compute increases than earlier scale-ups.
Counts as contradicted if: Comparable measurements across a representative set of capabilities show unchanged or greater gains for similar proportional compute increases.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
OpenAI reported capability improvements from scaling GPT-4.5 on February 27, 2025, but did not establish whether gains per proportional increase in training compute had declined [e5]. Epoch's retrospective analysis, available September 16, 2025, found GPT-4.5's gains on GPQA and Math Level 5 consistent with historical compute-scaling trends [e6]. This weighs against declining returns on those benchmarks, but these items do not provide the representative, controlled longitudinal comparison needed to settle the broader February 2025 claim [e5, e6].
Limitations
- The evidence does not isolate pretraining scale from changes in data, architecture, and post-training across representative capabilities.
- The claim specifies neither an effectiveness metric nor an earlier comparison period.
- The exact date of the February statement is unknown; the February 27 release cannot establish what was observable when it was made.
Evidence (2 items)
Goes against the claim: Epoch AI,
scaling roughly as expected.
Why it matters here: Epoch compares GPT-4.5's GPQA and Math Level 5 gains with historical gains per order of magnitude of compute and finds them consistent with the trend. This is material counterevidence on those benchmarks, but not a controlled comparison covering the full scope of effectiveness.
- Where
- Page 21, footnote 12
- Publisher
- Epoch AI
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: OpenAI,
capability improvements
Why it matters here: OpenAI reports capability gains from GPT-4.5 and supplies comparisons with GPT-4o. Continued gains alone do not establish whether returns to additional training compute had diminished.
- Where
- Section Safety
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
A longitudinal comparison of frontier scale-ups relevant to February 2025 and an explicit earlier baseline, using consistent capability tests and training-compute estimates, comparing gains for similar proportional compute increases while controlling for other training changes.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, OpenAI’s o3 and Tyler Cowen’s Misguided AGI Fantasy
my perspective that scaling was no longer working as effectively as it once did
- Where
- paragraph 30
- Published
- 2025-04-17
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-openais-o3-and-tyler-cowens-misguided-b9c3c2-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-openais-o3-and-tyler-cowens-misguided-b9c3c2-a2.