“An aside about software engineering: I actually believe LLMs have some value here.”
Our reading, the sentence that was checked:
LLMs have some value in software engineering.
Assessment: Supported Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, Sincerity Wins The War, paragraph 74.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-sic-0525e3-a1
We recorded 21 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-06-16. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- LLMs have some value
- software engineering
Our interpretation
- The ordinary reading is that LLM assistance provides practical benefits for at least some software engineering work. It does not assert universal productivity gains or autonomous replacement.
- For this ongoing state, I use June 2025 through the publication date as the latest reportable month-to-date period. Earlier experiments provide context.
Counts as supported if: Credible evidence reports a practical benefit from LLM-assisted software engineering during the target period.
Counts as contradicted if: Evidence establishes that the supposed benefits were spurious and that LLM assistance provided no practical software engineering value in the target period. Negative results for particular tasks alone are insufficient.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Google publicly reported a measured engineering productivity benefit before the post. An earlier randomized experiment also found faster completion of a development task. That is sufficient for the limited claim that LLMs have some software engineering value. It does not establish benefits for every engineer or task.
Limitations
- The current-period evidence is a company estimate.
- The controlled experiment concerns an earlier period and a particular task.
Evidence (2 items)
Supports the claim: Lex Fridman Podcast; statement by Google's CEO,
10%
Why it matters here: Pichai identifies the measure immediately beforehand as Google's engineering velocity improvement from AI, rather than the share of generated code. This contemporaneous company estimate supports the modest claim of some practical value.
- Publisher
- Lex Fridman Podcast; statement by Google's CEO
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: arXiv; Paradis and colleagues,
significantly shortened
Why it matters here: The randomized study involved 96 Google engineers and estimated about a 21% reduction in time on a particular task. Its summer 2024 observations corroborate the existence of benefits historically but fall outside the current observation window.
- Where
- Abstract
- Publisher
- arXiv; Paradis and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, Sincerity Wins The War
An aside about software engineering: I actually believe LLMs have some value here.
- Where
- paragraph 74
- Published
- 2025-06-16
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-sic-0525e3-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.