“It cannot do much more than it is currently doing, other than doing more of it faster with some new inputs.”
Our reading, the sentence that was checked:
Generative AI cannot do much more than it currently does, beyond doing it faster with new inputs.
Assessment: Contradicted Settled before its deadline on 2026-10-29 Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, Pop Culture, paragraph 60.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-pop-culture-a52dc0-a1
We recorded 27 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-07-08. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- Settled before its deadline
- Deadline state on 2026-10-29
- Settled before its deadline (the release’s reference date)
From the speaker
- cannot do much more
- than it is currently doing
- other than doing more of it faster
- with some new inputs
Our interpretation
- The surrounding forecast language makes this an open-ended claim about a capability ceiling relative to July 8, 2024.
- The plain reading excludes substantial gains in solving difficult tasks, not merely entirely new categories of task.
- Language models trained to generate reasoning remain within generative AI.
- Higher task-success rates can contradict the claim even if they require more computation; speed alone cannot.
Counts as supported if: A demonstrated binding capability limit establishes that substantial further gains are impossible. A finite period without gains would not suffice.
Counts as contradicted if: A later system demonstrates substantial improvements on comparable substantive tasks beyond the July 2024 baseline, rather than only faster generation or additional input formats.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Contradicted
The dated evidence goes against what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- Settled before its deadline
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The independent planning evaluation documents a substantial increase in problems solved, including obfuscated problems on which earlier models performed very poorly. That exceeds faster output or additional input formats and meets the early-resolution rule. The improvement was neither universal nor reliable on every harder task, but the claim excludes substantial capability growth, not just flawless performance.
Limitations
- The planning study is a preprint and uses bounded benchmark tasks.
- The researchers used GPT-4o-mini and a parser to normalise o1's output format.
- Performance deteriorated on longer problems, and the study did not establish general intelligence.
- The decisive date is the public report date, not an asserted date for individual test runs.
Evidence (2 items)
Goes against the claim: Valmeekam, Stechly, Gundawar and Kambhampati / arXiv,
correctly answers 97.8%
Why it matters here: The public evaluation reported 97.8% success on basic Blocksworld and 52.8% on its obfuscated version, where earlier tested LLMs remained below 5%. These are substantive planning gains; the event date denotes publication of the demonstration, since exact test-run dates are not supplied.
- Publisher
- Valmeekam, Stechly, Gundawar and Kambhampati / arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: ARC Prize,
time cost.
Why it matters here: ARC Prize found o1-preview approximately matched Claude 3.5 Sonnet's accuracy on its public evaluation while taking much longer. This limits the breadth of the improvement but does not negate the planning gains.
- Publisher
- ARC Prize
- Kind of source
- Independent measurement
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, Pop Culture
It cannot do much more than it is currently doing, other than doing more of it faster with some new inputs.
- Where
- paragraph 60
- Published
- 2024-07-08
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-pop-culture-a52dc0-a1: Current. Outcome: Contradicted. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.