“they're allowed to get away from the very obvious problem: that Large Language Models are peaking, will not solve the kind of complex problems that actually matter, and OpenAI (and other LLM companies) are being allowed to accumulate money and power in a way that's allowed them to do actual damage in broad daylight.”
Our reading, the sentence that was checked:
LLMs are peaking in capability.
Assessment: Unresolved Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, Put Up Or Shut Up, paragraph 35.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-put-up-or-shut-up-7d0345-a2
We recorded 19 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-07-16. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- Large Language Models
- are peaking
Our interpretation
- The plain reading is that frontier LLM capability gains were approaching a plateau around publication.
- Use April–June 2024, the latest completed quarter before the post, to assess the ongoing state. Earlier model generations supply comparison baselines.
- Peaking does not necessarily mean that every benchmark score has stopped improving. Later breakthroughs cannot by themselves disprove this historical observation.
Counts as supported if: Comparable independent evaluations across several capability domains show sustained flattening of frontier performance over successive model generations through the target period.
Counts as contradicted if: Comparable independent evaluations show substantial, broad and sustained frontier capability gains through the target period, inconsistent with an approaching plateau.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The June 2024 LiveBench evaluation found Claude 3.5 Sonnet substantially ahead of other tested models in reasoning and coding [e1]. The June 2024 OlympicArena evaluation found a clear performance gap favoring GPT-4o and Claude 3.5 Sonnet over Gemini 1.5 Pro and GPT-4V [e2]. These comparisons weigh against stagnation, but establish neither sustained flattening nor substantial, broad and sustained gains through April–June 2024 [e1, e2]. The evidence therefore does not settle whether capabilities were approaching a plateau [e1, e2].
Limitations
- The evaluations compare selected models and tasks; they do not establish the broader rate of capability improvement over successive generations.
- The evidence does not supply a threshold distinguishing slowing progress from an approaching plateau.
Evidence (2 items)
Goes against the claim: White et al., arXiv,
substantially outperforms all other models
Why it matters here: The June evaluation documents meaningful gains by a newly released frontier model. It weighs against a plateau, but does not establish a sustained industry-wide rate of improvement.
- Publisher
- White et al., arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Goes against the claim: Huang et al., arXiv,
clear performance gap
Why it matters here: Another contemporary evaluation finds a performance gap favoring newer frontier models. This is counterevidence to stagnation, but a ranking snapshot cannot identify a long-term ceiling.
- Publisher
- Huang et al., arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Comparable evaluations across successive frontier model generations and several capability domains through June 2024, using consistent scoring and a defensible threshold to distinguish sustained flattening from substantial continuing gains.
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, Put Up Or Shut Up
they're allowed to get away from the very obvious problem: that Large Language Models are peaking, will not solve the kind of complex problems that actually matter, and OpenAI (and other LLM companies) are being allowed to accumulate money and power in a way that's allowed them to do actual damage in broad daylight.
- Where
- paragraph 35
- Published
- 2024-07-16
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-put-up-or-shut-up-7d0345-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.