LLMs + Coding Agents = Security Nightmare
We recorded 9 claims from this post. 2 were checked against evidence. 7 can’t be checked. 7 more are held. Read the post at its source.
Claims from this post (9)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Forecast
“(Nvidia advised, rightly, that everyone should disable Auto-Run mode if they’ve activated it, but we fear that many users may use it anyway, because it is so fast.)”
Our reading: Many Cursor users may keep using Auto-Run mode despite the security risk because it is fast.
No horizon
Qualifier: because it is so fast
Assessment: Can’t be checkedSaid
Observation
“the user has the option to accept a code change or not, but with so many changes to make at such a rapid pace, a developer in hurrry (which is almost every developer) could easily miss an attack, and ok the change or run a command”
Our reading: Even with manual approval, hurried developers could easily approve malicious changes suggested by coding agents.
No horizon
Qualifier: even with Auto-Run off
Assessment: Can’t be checkedSaid
Forecast
“Given current implementations of the technology, it's hard to imagine enough patches on the planet to thwart them all.”
Our reading: Given current implementations, it is unlikely that patches can thwart all data-injection attacks on generative AI systems.
No horizon
Qualifier: given current implementations
Assessment: Can’t be checkedSaid
Observation
“As long as we have agents roaming the internet and otherwise incorporating data that they don’t fully understand – and LLMs don’t ever fully understand the data they are leveraging – there is enormous risk.”
Our reading: LLMs never fully understand the data they leverage.
No horizon
Assessment: UnresolvedSaid
Causal claim
“Agency plus LLMs have already led multiple “vibe coders” to lose databases, with new reports coming in every day or two, stemming simply from the unreliability inherent in LLMs (that Gary has so often harped on).”
Our reading: Agency combined with LLMs has already led multiple vibe coders to lose databases, with new reports every day or two, and these losses stem simply from the unreliability built into LLMs.
No horizon Evidence as of
Limitation: The Cursor incident rests on a user allegation without execution logs or an independent postmortem [e2].
Assessment: UnresolvedSaid
Forecast
“The bad news is that although this one was stopped, many others won’t be. No one patch would be enough, or even a thousand; there are just too many variations on the overall theme”
Our reading: Many attacks on AI coding tools won't be stopped; not even a thousand patches would suffice given the number of variations.
No deadline stated Evidence as of
Limitation: The evidence concerns researcher demonstrations rather than a census of malicious attacks.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The best defense would be not using agentic coding altogether.”
Our reading: The best security defense is to not use agentic coding at all.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“But the tools are so seductive that we doubt many developers will resist.”
Our reading: Few developers are likely to refrain from using agentic coding tools.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“Sure the advice is good, as far is it goes, but if the goods inside are valuable enough, motivated thieves may well still find a way.”
Our reading: Even with mitigations, motivated attackers may still compromise valuable systems using coding agents.
No horizon
Only if: even with mitigations
Held from this post (7)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: Two new technologies, LLMs and coding agents, are radically increasing the attack surface, meaning the space for potential vulnerabilities.
- Attribution is uncertain
- The two reviews read the claim differently
- Attribution is uncertain
- Attribution is uncertain
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: Many developers will find it hard to resist giving AI tools far more access and permissions than they should, drawn by hopes of convenience and productivity.