“Newton is also conveniently ignoring an absolutely fundamental distinction, between early period of reliable, rapid progress from 2019-2022 (when training for GPT-4) was completed, and a more recent regime in which by many measures, progress has (reportedly) slowed.”
Our reading, the sentence that was checked:
After rapid progress in 2019-2022, LLM progress has by many measures reportedly slowed.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Casey (Newton) Strikes Out, paragraph 9.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-casey-newton-strikes-out-500024-a1
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-12-11. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- by many measures (qualifier, stated by the speaker)
- reportedly (qualifier, stated by the speaker)
- since GPT-4 training completed (qualifier, stated by the speaker)
From the speaker
- reliable, rapid progress from 2019-2022
- a more recent regime
- by many measures
- progress has (reportedly) slowed
Our interpretation
- The ordinary reading is that several indicators reportedly showed slower LLM improvement after 2022, compared with 2019–2022.
- The word 'reportedly' preserves uncertainty. Establishing that reports existed supports that qualification, but does not by itself establish the full comparison across many measures.
- I use 2023-01-01 through the post date for the recent period. The earlier years provide the comparison baseline.
- Slower progress does not mean no progress. Diminishing returns from pretraining and the pace of overall capability improvement are related but different measurements.
Counts as supported if: Credible reports or evaluations document smaller improvements across several distinct capability or scaling measures in the recent period, with a meaningful comparison against the earlier period.
Counts as contradicted if: Comparable evidence across several measures establishes that the alleged slowdown was mistaken and that the recent pace matched or exceeded the earlier pace. Isolated improvements are insufficient.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Credible late-2024 reports did describe disappointing pretraining gains, so Marcus's reference to reports has support. The broader comparison remains unsettled: those reports do not provide consistent measurements of the earlier and later pace across several measures. Anthropic disputed deviations from scaling laws, and OpenAI reported substantial reasoning gains. Neither continued improvement nor disappointing private experiments alone settles the claimed change in pace.
Limitations
- A narrower assessment asking only whether slowdown reports existed would be supported.
- Company evaluations and statements are not independent measurements.
- Later model improvements cannot by themselves establish what the pace was before this post.
Evidence (3 items)
Supports the claim: Reuters, republished by Yahoo Finance,
"bigger is better" philosophy.
Why it matters here: Reuters reported Sutskever's account of plateauing pretraining returns and sources describing disappointing efforts to surpass GPT-4. This supports the existence of credible slowdown reports, but supplies no public, comparable multi-measure series; news reporting is necessary because the underlying experiments were private.
- Publisher
- Reuters, republished by Yahoo Finance
- Kind of source
- News report
- Describes events of
- Public from
Context only: TIME,
Recent releases paint a somewhat mixed picture.
Why it matters here: TIME obtained an Anthropic statement denying observed deviations from scaling laws and discussed differing results across releases. This challenges treating the reports as an established field-wide finding, without itself proving an unchanged rate.
- Publisher
- TIME
- Kind of source
- News report
- Describes events of
- Public from
Context only: OpenAI,
o1 significantly outperforms GPT‑4o
Why it matters here: OpenAI disclosed substantial gains on reasoning evaluations using a different training and inference approach. These gains limit generalization from pretraining difficulties, but do not alone settle a comparison of progress rates.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
A dated, comparable evaluation series across several capability or scaling measures covering both periods, or sufficiently detailed records substantiating the reported comparisons.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Casey (Newton) Strikes Out
Newton is also conveniently ignoring an absolutely fundamental distinction, between early period of reliable, rapid progress from 2019-2022 (when training for GPT-4) was completed, and a more recent regime in which by many measures, progress has (reportedly) slowed.
- Where
- paragraph 9
- Published
- 2024-12-11
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-casey-newton-strikes-out-500024-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.