“To his credit, Shumer is actually correct that something has changed recently. You really can let things rip more in the most recent systems.”
Our reading, the sentence that was checked:
Recent AI coding systems have changed: you can let them run more on their own than before.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, About that Matt Shumer post that has nearly 50 million views, paragraph 9.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-about-that-matt-shumer-post-that-da51c2-a3
We recorded 6 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-02-11. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- something has changed recently
- You really can let things rip more
- in the most recent systems
Our interpretation
- 'Recently' is read as the few months before the post, roughly late 2025 to early February 2026.
- 'Let things rip more' is read as: recent coding agents can work on longer tasks with less human help than earlier systems.
Counts as supported if: Measurements or credible disclosures from late 2025 to early 2026 show recent models handling longer autonomous coding tasks than their predecessors.
Counts as contradicted if: Measurements from the period show no meaningful gain in autonomous task length or success for the newest models over earlier ones.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Marcus says recent systems can be left to run more on their own. On 19 December 2025, METR estimated that Claude Opus 4.5 had a 50% time horizon of about 4 hours 49 minutes on its tasks. METR called this its highest published time horizon to date (e1). On 5 February 2026, Anthropic said Opus 4.6 'sustains agentic tasks for longer' than its predecessor (e2). This is a company claim, not a measurement. The record METR figure was at the 50% success level. At the 80% level, Opus 4.5's time horizon was 27 minutes, which METR said was similar to past models and below GPT-5.1-Codex-Max's 32 minutes (e1). A LessWrong commentary described the 80% horizon as essentially flat since GPT-5.1-Codex-Max (e3). So the evidence shows longer tasks completed at the 50% level, but not a matching gain in reliable completion.
Limitations
- 'Let things rip more' is informal and has no threshold, so it is judged against task-length measures.
- The METR figures come from a LessWrong reproduction of METR's thread. The 95% confidence interval for the 50% horizon is wide, from 1 hr 49 mins to 20 hrs 25 mins.
- Anthropic's statement about Opus 4.6 is a company disclosure. METR measurements of Opus 4.6 were published after the post and are not used here.
Evidence (3 items)
Supports the claim: LessWrong (quoting METR on X),
Claude Opus 4.5 has a 50%-time horizon of around 4 hrs 49 mins (95% confidence interval of 1 hr 49 mins to 20 hrs 25 mins). While we're still working through evaluations for other recent models, this is our highest published time horizon to date.
Why it matters here: METR measured a record jump in autonomous task length for a late-2025 model. That shows recent systems can be left to run on longer tasks.
- Publisher
- LessWrong (quoting METR on X)
- Kind of source
- Secondary summary
- Describes events of
- Public from
Supports the claim: Anthropic,
It plans more carefully, sustains agentic tasks for longer, can operate more reliably in larger codebases
Why it matters here: The developer says its newest model sustains agentic tasks for longer. This is a company claim and agrees with METR's trend.
- Publisher
- Anthropic
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: LessWrong,
The 80%-horizon has stayed essentially flat (27-32 mins) since GPT-5.1-Codex-Max's release but there's a big jump with huge error bars on the 50%-horizons.
Why it matters here: The gains were concentrated at the 50% success level, which fits Marcus's caveat that results come out right only 'sometimes'.
- Publisher
- LessWrong
- Kind of source
- Secondary summary
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, About that Matt Shumer post that has nearly 50 million views
To his credit, Shumer is actually correct that something has changed recently. You really can let things rip more in the most recent systems.
- Where
- paragraph 9
- Published
- 2026-02-11
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-about-that-matt-shumer-post-that-da51c2-a3: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-about-that-matt-shumer-post-that-da51c2-a3.
HeldBy this site
The final-text check failed: rationale_supported_by_evidence; overclaims: The gain was limited to the 50% success level..
About assessment marcus-about-that-matt-shumer-post-that-da51c2-a2, which was not published.