“Secure, trustworthy AI could be a great thing, but LLMs will never be that.”
Our reading:
LLMs will never be secure, trustworthy AI.
Assessment: Unresolved
What happened
Did the forecast come true?
Outcome: Unresolved
We checked, and the evidence so far doesn’t settle it.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The March 4, 2025 study version reports adaptive attacks bypassing all eight evaluated defenses, with success rates above 50% [e3]. Those results establish weaknesses in the tested defenses, not impossibility for every future LLM system [e3]. CaMeL's June 24, 2025 version describes protections alongside policy-maintenance burdens, excluded attacks, and potential side channels [e4]. LLMbda's July 10, 2026 version provides machine-checked information-flow guarantees, with modeling limits, policy relaxations, and two successful attacked runs in its evaluation [e5]. Neither defense supplies the broad independent evaluation required for a conclusive counterexample, and the documented attacks cannot establish an unlimited 'never' forecast [e3, e4, e5].
Caveats
- The attack study covers eight defenses and cannot establish the security of all present or future systems [e3].
- The candidate defenses do not establish overall security, output reliability, and retained usefulness across realistic deployment threats [e4, e5].
- The supplied evaluations do not meet the contract's independent evaluation requirement for early resolution [e4, e5].
Evidence (3 items)
Context only: arXiv; Qiusi Zhan and coauthors,
This reveals critical vulnerabilities in current defenses.
Why it matters: The researchers bypassed all eight evaluated defenses with adaptive attacks, reporting success rates above 50%. This documents weaknesses in those defenses, but cannot establish that every future LLM will remain insecure.
- Publisher
- arXiv; Qiusi Zhan and coauthors
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Context only: arXiv; Edoardo Debenedetti and coauthors,
No, prompt injection attacks are not fully solved.
Why it matters: CaMeL provides control-flow and data-flow protections around vulnerable models. Its authors also identify excluded text-manipulation attacks, policy-maintenance burdens, and possible side channels, so this proposal does not meet the broader early-resolution rule.
- Publisher
- arXiv; Edoardo Debenedetti and coauthors
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Context only: arXiv; Zac Garby, Andrew D. Gordon, and David Sands,
must be probabilistic
Why it matters: LLMbda supplies machine-checked information-flow guarantees and a banking-agent evaluation, providing material progress against the categorical pessimism. Its stated modeling limits, policy relaxations, and two successful attacked runs prevent treating it as independent establishment of overall security and trustworthiness.
- Publisher
- arXiv; Zac Garby, Andrew D. Gordon, and David Sands
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
How we judged it
- Kind of claim
- Forecast
- Deadline
- “never”
Where we had to interpret
- In context, secure and trustworthy means dependable operation against adversarial prompting, including jailbreaks, unauthorized actions, and disclosure of protected information.
- I use practical security for a meaningful intended deployment, rather than requiring perfection against every conceivable threat.
- A protected LLM-based system can be a candidate counterexample, but a guarantee for one narrow property does not by itself establish overall security and trustworthiness.
- Never supplies no finite deadline. Continued failures cannot establish the forecast for all future LLMs.
Right if: A general impossibility result would have to rule out practically secure and trustworthy LLM-based systems across the claim's scope. A finite collection of failed systems cannot establish never.
Wrong if: A publicly documented LLM-based system receives credible independent evaluation establishing dependable security and trustworthiness in a meaningful deployment against realistic jailbreak and prompt-injection threats.
We wrote this test after the outcome was known.
What would settle it
A dated, reproducible independent evaluation establishing practical security and trustworthiness in a useful LLM deployment would contradict the forecast. It must test realistic adaptive attacks and cover confidentiality, unauthorized actions, output reliability, and retained usefulness beyond one property or benchmark. A general impossibility result covering the claim's full scope would support it.
We found 7 claims in this post. See them all.
Full record
Source and dates
Marcus on AI, An unending array of jailbreaking attacks could be the death of LLMs, paragraph 8.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-29. The post may have been edited after 2024-04-02.
Where it was said (1)
Said once in the posts we read.
: Marcus on AI, An unending array of jailbreaking attacks could be the death of LLMs
Secure, trustworthy AI could be a great thing, but LLMs will never be that.
- Where
- paragraph 8
- Published
- 2024-04-02
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Unresolved. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
The final check failed: a cited source didn’t say what the write-up said.