“No system solved more than 4 (or maybe any) of the Marcus-Brundage tasks.”
Our reading:
In 2025 no AI system solved more than 4, and possibly none, of the Marcus-Brundage tasks.
Assessment: Unresolved
What happened
Was it true for the period described?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Evidence up to . Checked .
Why
The original wager, published December 30, 2024, sets an adjudication framework and allows limited adaptation between tasks; it does not report task completions [e1]. A retrospective assessing year-end 2025, available January 2, 2026, calls the upper-bound prediction correct and suggests possible reading-comprehension success [e2]. That retrospective supplies no underlying task-level tests and does not isolate results through December 20, 2025 [e2]. It therefore offers limited support without settling the claim [e2].
Caveats
- The packet lacks a comprehensive independent assessment of qualifying systems against all ten tasks.
- It contains no documented five-task completion by one qualifying system. That absence does not prove none occurred.
- The year-end retrospective does not establish which results were achieved during January 1–December 20, 2025.
- The hedged possibility of zero successes is not assessed as a firm assertion.
- The original wager's 2027 deadline does not govern this historical observation.
Evidence (2 items)
Context only: Gary Marcus and Miles Brundage, Marcus on AI,
with broad discretion.
Why it matters: The joint original statement establishes that these are demanding tasks with an adjudication framework, rather than a standard benchmark leaderboard. It supplies definitions only and is not evidence that Marcus's subsequent count was correct.
- Publisher
- Gary Marcus and Miles Brundage, Marcus on AI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- About events on
- Published
Context only: jessicata, LessWrong via GreaterWrong,
AI can perhaps pass the reading comprehension task.
Why it matters: The retrospective judges the upper-bound prediction correct while allowing possible success on reading comprehension. It assesses year-end and supplies neither a system-by-system audit nor primary demonstrations, so it is not decisive for the December 20 observation.
- Publisher
- jessicata, LessWrong via GreaterWrong
- Kind of source
- Secondary summary
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- The firm assertion is that no single system completed at least five of the ten tasks by December 20, 2025.
- The parenthetical possibility of zero is hedged and is not treated as a firm zero-success assertion.
- The retrospective concerns 2025 through publication, not the original wager's 2027 deadline or the remaining days of 2025.
- Success must satisfy the tasks' full requirements, including reliability where specified. Related benchmark scores alone do not count.
Right if: A sufficiently comprehensive independent task-level assessment establishes that no qualifying system completed five tasks during the target period.
Wrong if: Dated, independently assessable results establish that one qualifying system completed at least five tasks during the target period.
We wrote this test after the outcome was known.
What would settle it
A comprehensive independent assessment establishing that no qualifying system completed at least five tasks during January 1–December 20, 2025, or dated, independently assessable results showing that one qualifying system did. The assessment must preserve each task's full requirements and identify which results belong to the same system.
We found 15 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, Six (or seven) predictions for AI 2026 from a Generative AI realist, paragraph 1.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2025-12-20.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, Six (or seven) predictions for AI 2026 from a Generative AI realist
No system solved more than 4 (or maybe any) of the Marcus-Brundage tasks.
- Where
- paragraph 1
- Published
- 2025-12-20
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
The final check failed: a cited source didn’t say what the write-up said.