An unending array of jailbreaking attacks could be the death of LLMs
We recorded 5 claims from this post. 5 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (5)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Observation
“There seems to be an endless number of ways of attacking them.”
Our reading: There appears to be an endless number of ways to attack LLMs.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Some get patched, but there very likely will aways be more.”
Our reading: Even as some jailbreaks are patched, there will very likely always be more jailbreak attacks on LLMs.
No horizon
Qualifier: even though some get patched
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“That would be fine if LLMs were kept in the lab, where they arguably still belong”
Our reading: LLMs arguably still belong in the lab rather than in widespread deployment.
No horizon
Qualifier: arguably
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“That would be fine if LLMs were kept in the lab, where they arguably still belong, but with hundreds of millions of people using them daoly, and companies like OpenAI promising to put them in all purpose agents with essentially full device-level access, the lack of any kind of guarantee gets more worrisome by the day.”
Our reading: With hundreds of millions of people using LLMs daily and companies like OpenAI promising to put them in all-purpose agents with essentially full device-level access, the lack of any kind of guarantee about LLMs gets more worrisome by the day, though it would be fine if LLMs were kept in the lab, where they arguably still belong.
No horizon
Qualifier: hundreds of millions of people using LLMs daily
Assessment: Can’t be checkedSaid
Observation
“As noted above, Rule 1 of cybersecurity is to keep your attack surface small; in LLMs the attack surface appears to be infinite. That can’t be good.”
Our reading: The attack surface of LLMs appears to be infinite, violating the cybersecurity principle of keeping attack surfaces small.
No horizon
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: Because nobody knows exactly how LLMs work, nobody can ever issue strong guarantees about them.
- The two reviews read the claim differently