“The LRMs failed on Tower of Hanoi with 8 discs, where the optimal solution is 255 moves, well within so-called token limits;”
Our reading, the sentence that was checked:
The LRMs failed Tower of Hanoi with 8 discs, whose optimal solution of 255 moves is well within so-called token limits, so the token-limit objection does not explain the overall pattern of results.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Seven replies to the viral Apple reasoning paper – and why they fall short, paragraph 6.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-seven-replies-to-the-viral-apple-b26773-a2
We recorded 17 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-06-12. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- Tower of Hanoi in the Apple paper (scope, stated by the speaker)
- the overall pattern of results (scope, stated by the speaker)
From the speaker
- The LRMs failed on Tower of Hanoi with 8 discs
- the optimal solution is 255 moves
- well within so-called token limits
Our interpretation
- Failed means the tested models exhibited execution failures at eight discs; it does not require every model to fail every sampled attempt.
- Within token limits refers to the required complete answer fitting the configured output budget. It does not guarantee that every possible reasoning trace fits.
- The contextual conclusion is that a hard output ceiling cannot explain all reported failures, not that output constraints never matter.
- The target is the original Apple experiment reported in June 2025.
Counts as supported if: The original results show eight-disc failures, and a valid 255-move answer fits comfortably within the tested output budgets.
Counts as contradicted if: The original results show no eight-disc failures, the optimal move count differs, or the required answer necessarily exceeds the applicable output budgets.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
In a November 20, 2025 retrospective account of the June 2025 experiment, the Apple researchers report eight-disc Hanoi failures around move 40 of the required 255 moves [e1]. The corrected June 16, 2025 rebuttal estimates about 10 tokens per move: roughly 2,550 tokens for the final move list, well below the discussed 64,000- and 100,000-token budgets [e2]. This supports the narrow conclusion that a hard output ceiling cannot explain all reported failures [e1, e2].
Limitations
- The token estimate excludes setup, formatting overhead, and reasoning tokens.
- The evidence does not establish the precise cause of every failure or exclude premature stopping.
- Support does not imply that every model failed every eight-disc attempt or validate results for other puzzles.
Evidence (3 items)
Supports the claim: arXiv; Shojaee and colleagues,
failures
Why it matters here: The authors’ retrospective account of the original study reports errors around move 40 for eight-disc puzzles requiring 255 moves. This directly supports failures occurring before the required answer approaches the output ceiling.
- Publisher
- arXiv; Shojaee and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: arXiv; A. Lawsen,
well before these theoretical limits.
Why it matters here: The corrected rebuttal estimates ten tokens per move and discusses budgets of 64,000 and 100,000 tokens. Its own calculation places the required eight-disc move list comfortably below those budgets, while retaining premature stopping as an alternative explanation.
- Publisher
- arXiv; A. Lawsen
- Kind of source
- Preprint, not peer reviewed
- Arithmetic
- 2^8 - 1 = 255 moves; 255 × 10 ≈ 2,550 answer tokens, excluding setup and reasoning; 2,550 / 64,000 ≈ 3.98%. (inputs: Eight discs; Minimum move count: 2^N - 1; Approximately 10 tokens per move; 64,000-token budget; Move count is exact. Token count is an estimate; budget percentage is rounded to two decimal places.)
- Describes events of
- Public from
Context only: arXiv; Dellibarda Varela and colleagues,
strictly to solvable problems
Why it matters here: Later experiments identify a solvability problem in another puzzle family. This limits broad conclusions about the entire study, but does not negate the original eight-disc Hanoi failures.
- Publisher
- arXiv; Dellibarda Varela and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Seven replies to the viral Apple reasoning paper – and why they fall short
The LRMs failed on Tower of Hanoi with 8 discs, where the optimal solution is 255 moves, well within so-called token limits;
- Where
- paragraph 6
- Published
- 2025-06-12
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-seven-replies-to-the-viral-apple-b26773-a2: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.