“The new paper very much amplifies what a team of AWS and Berkeley researchers showed in February, documenting similar vulnerabilities:”
Our reading, the sentence that was checked:
The new findings on agent vulnerabilities reinforce similar vulnerabilities documented by AWS and Berkeley researchers in February 2026.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Breaking: Autonomous Agents are a Shitshow, paragraph 6.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-breaking-autonomous-agents-are-a-b1d87b-a1
We recorded 4 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-05-05. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- a team of AWS and Berkeley researchers
- showed in February
- documenting similar vulnerabilities
- the new paper very much amplifies
Our interpretation
- Holds if a paper by AWS and UC Berkeley researchers, released or revised in February 2026, documented agent vulnerabilities similar to the new study's, such as multi-step tool-chaining attacks.
- 'Amplifies' is read as the new study reporting consistent findings at larger or production scale. It is not a strict causal claim.
Counts as supported if: An AWS/Berkeley paper dated February 2026 reports high agent vulnerability to tool-chaining or similar attacks, and the new study reports similar vulnerabilities.
Counts as contradicted if: No such AWS/Berkeley paper exists from around February 2026, or its findings concern unrelated vulnerabilities.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The STAC paper by AWS AI Labs and UC Berkeley researchers was revised on arXiv on 2 February 2026. That version reports that agents including GPT-4.1 are highly vulnerable to sequential tool-chaining attacks, with success rates above 90% in most cases. The new Elloe-led study reports a similar 91% tool-chaining vulnerability. GovInfoSecurity says the AWS/UC Berkeley researchers documented the same dynamic in an earlier February paper. One small caveat: STAC first appeared in September 2025, so 'February' refers to its revised version.
Limitations
- Marcus does not name the February paper; it is identified as STAC by inference.
- STAC first appeared in September 2025, not February 2026.
- 'Amplifies' is a qualitative judgment.
Evidence (3 items)
Supports the claim: arXiv,
Our evaluations show that state-of-the-art LLM agents, including GPT-4.1, are highly vulnerable to STAC, with attack success rates (ASR) exceeding 90% in most cases.
Why it matters here: The February 2026 version (v2) of the STAC paper documents high agent vulnerability to sequential tool-chaining attacks, the same class of vulnerability as the new study's headline finding.
- Where
- Abstract
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: GovInfoSecurity (ISMG),
Amazon Web Services AI Labs and UC Berkeley researchers documented the same dynamic in an earlier February paper.
Why it matters here: An independent report links the new study to the February AWS/UC Berkeley paper and says the two found the same dynamic.
- Where
- paragraph 5
- Publisher
- GovInfoSecurity (ISMG)
- Kind of source
- News report
- Describes events of
- Public from
Context only: arXiv,
The work was done during an AWS AI Labs internship. Affiliation: UC Berkeley
Why it matters here: Shows the STAC team: the first author is from UC Berkeley and the other authors are listed with AWS AI Labs.
- Where
- author list
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Breaking: Autonomous Agents are a Shitshow
The new paper very much amplifies what a team of AWS and Berkeley researchers showed in February, documenting similar vulnerabilities:
- Where
- paragraph 6
- Published
- 2026-05-05
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-breaking-autonomous-agents-are-a-b1d87b-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-breaking-autonomous-agents-are-a-b1d87b-a1.