“As I wrote a few weeks ago, LLMs are good at writing a lot of code, not good code, and the more people you allow to use them, the more code you’re going to generate, which means the more time you’re either going to need to review that code, or the more vulnerabilities you’re going to create as a result.”
Our reading, the sentence that was checked:
LLMs write a lot of code, not good code; broader use means more review time or more vulnerabilities.
Assessment: Unresolved Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, AI Is Really Weird, paragraph 90.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-ai-is-really-weird-1aca15-a1
We recorded 42 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-04-08. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- good at writing a lot of code, not good code
- the more people you allow to use them
- the more code you’re going to generate
- more time you’re either going to need to review that code
- more vulnerabilities
Our interpretation
- The plain reading is a general causal claim about expanding LLM-assisted development: output increases, while maintaining quality requires more total review effort or accepting more vulnerabilities.
- Good code means code that is correct, maintainable and secure in ordinary development work.
- Review time means total review labor, not merely elapsed time per change. Vulnerabilities means actual security weaknesses, not all warnings from a code analyzer.
- The claim describes a general mechanism, not a forecast with a deadline. Historical experiments can test parts of that mechanism, but older tools cannot establish its prevalence in April 2026.
Counts as supported if: Credible comparisons show that broader LLM adoption increases code output and causes additional review labor or security vulnerabilities, with evidence that the generated code generally has quality shortcomings.
Counts as contradicted if: Credible comparisons across relevant development settings show increased output without additional total review labor or vulnerabilities, and maintained or improved code quality.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
There is experimental evidence that an AI assistant can worsen security. There is also controlled evidence finding no detectable downstream maintenance penalty. Neither establishes the full claim that broader use generally forces more review labor or more vulnerabilities. The findings concern different tasks and outcomes, so combining them into a decisive verdict would overstate what they show.
Limitations
- Security tasks and maintenance tasks measure different aspects of quality.
- A failure to detect a difference does not prove equal quality.
- The cited experiments do not represent every model or development setting.
Evidence (2 items)
Context only: Perry et al.; ACM CCS, author manuscript on arXiv,
less secure solutions
Why it matters here: This experiment supports the possibility that AI assistance worsens security. It tested an older assistant on bounded tasks and did not measure the effect of broader adoption on total code output and review labor.
- Publisher
- Perry et al.; ACM CCS, author manuscript on arXiv
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Context only: Borg et al.; arXiv,
no significant differences
Why it matters here: The maintenance experiment found no detectable disadvantage when developers subsequently modified AI-assisted code. This weighs against a blanket quality judgment, but neither proves equivalence nor measures security and total review labor.
- Publisher
- Borg et al.; arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
A controlled adoption study measuring code output, total review hours and independently verified vulnerabilities together across representative teams and tasks.
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, AI Is Really Weird
As I wrote a few weeks ago, LLMs are good at writing a lot of code, not good code, and the more people you allow to use them, the more code you’re going to generate, which means the more time you’re either going to need to review that code, or the more vulnerabilities you’re going to create as a result.
- Where
- paragraph 90
- Published
- 2026-04-08
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-ai-is-really-weird-1aca15-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.