“They’re not replacing anyone, and there isn’t compelling evidence that AI models speed people up. Allowing non-technical people to use LLMs to write code isn’t speeding up the delivery of software in a measurable way”
Our reading:
AI is not replacing workers, there is no compelling evidence it speeds people up, and LLM coding by non-technical people does not measurably speed software delivery.
Assessment: Contradicted
What happened
Was it true for the period described?
Outcome: Contradicted
The evidence goes against what they said.
Judged on a plain reading. They named no measure, so we used the reading a typical informed reader would take. It is spelled out under “How we judged it”.
Evidence up to . Checked .
Why
The broad denial of compelling productivity evidence is contradicted under the stated ordinary reading. Before the post, a randomized writing experiment found faster completion with better quality, and a peer-reviewed workplace study found higher measured output per hour. Earlier studies count because Zitron denied that such evidence existed. This finding does not establish that AI speeds everyone up: METR found a slowdown in one experienced-developer setting. The workforce assertion remains insufficiently established by the reviewed records, and the specific assertion about non-technical users delivering software remains unresolved. The combined claim is contradicted through its productivity-evidence assertion; this is not a finding against all three assertions.
Caveats
- Compelling is an evaluator-interpreted threshold.
- The writing experiment concerns bounded tasks, and the workplace study concerns one firm's support operation.
- Evidence of heterogeneous effects does not establish the blanket denial, but prevents a universal productivity conclusion.
- The workforce and non-technical software-delivery assertions are not conclusively graded.
Evidence (5 items)
Goes against the claim: Shakked Noy and Whitney Zhang, MIT,
productivity
Why it matters: This preregistered experiment randomly exposed half of 444 professionals to ChatGPT and found faster writing with improved quality. The event date marks the public existence of this evidence, the fact tested by the evidence-existence assertion, rather than the experiment's execution date.
- Where
- Abstract, page 1
- Publisher
- Shakked Noy and Whitney Zhang, MIT
- Kind of source
- Preprint, not peer reviewed
- About events on
- Published
Goes against the claim: The Quarterly Journal of Economics,
individual workers
Why it matters: The published study uses staggered deployment and comparison-based analysis of 5,172 support agents, finding about 15% more issues resolved per hour, with differing effects across workers. Its publication establishes that substantial workplace evidence existed before the post; it does not establish economy-wide gains.
- Publisher
- The Quarterly Journal of Economics
- Kind of source
- Peer-reviewed paper
- About events on
- Published
Context only: Klarna Group plc,
2,831
Why it matters: The adjacent passage attributes reductions to a strategy involving AI and other efficiency measures. This is relevant evidence against the blanket replacement assertion, but headcount changes and management attribution do not independently identify which positions AI replaced.
- Publisher
- Klarna Group plc
- Kind of source
- Regulatory filing
- About events on
- Published
Context only: METR,
19% longer
Why it matters: Random assignment across 246 issues completed by 16 experienced maintainers produced a slowdown. This limits generalization from positive studies, but neither erases their existence nor tests non-technical users delivering software.
- Publisher
- METR
- Kind of source
- Independent measurement
- About events on
- Published
Context only: METR,
some evidence for speedup
Why it matters: Before Zitron's post, METR had reported follow-up estimates suggesting speedup while warning that selection effects made the magnitude unreliable. The earlier slowdown should therefore not be presented as a settled description of current coding tools.
- Publisher
- METR
- Kind of source
- Independent measurement
- About events on
- Published
How we judged it
- Kind of claim
- Observation
- Deadline
- None given. We don’t add one.
Where we had to interpret
- Assess three assertions separately within this combined claim. Contradicting one does not establish that the other two are false.
- The relevant state is what had happened, and what evidence existed, by June 15, 2026. The evidence-existence assertion includes earlier studies; it does not describe only the latest quarter. The target period therefore has no stated lower bound.
- For the ongoing workforce and software-delivery assertions, use the latest relevant observations publicly reportable before the post. These subjects have no common reporting quarter, so no uniform quarterly window is imposed.
- The ordinary reading of replacing workers is that AI takes over work previously requiring people and causes positions to disappear or remain unfilled. Equivalent workload estimates and unrelated layoffs alone are insufficient.
- Read compelling evidence as credible controlled measurement of faster work or greater output per unit of time, with attention to output quality. This is an evaluator-supplied standard: Zitron supplied none.
- The broad productivity assertion does not require gains in every occupation or economy-wide productivity growth. Credible evidence of gains in actual work or realistic work tasks can contradict the denial.
- The software assertion specifically concerns people without technical programming competence and delivery of usable software. Faster code generation, professional-programmer experiments, and demonstrations without a comparison do not independently settle it.
Right if: Support requires evidence sustaining all three assertions: no demonstrated workforce substitution in the relevant organizational evidence, no credible measured productivity gains in the evidence available by the post, and adequate comparisons finding no measurable delivery improvement from non-technical users' LLM coding. Failure to find a study alone does not establish no effect.
Wrong if: A documented instance of actual workforce substitution, credible controlled evidence of faster work with acceptable quality, or a sound comparison showing faster usable-software delivery by non-technical LLM users contradicts the corresponding assertion. Identify which assertion is contradicted and leave untested assertions unsettled.
We wrote this test after the outcome was known.
What would settle it
For the remaining assertions, obtain records linking identified position reductions to AI taking over the work, and comparisons measuring usable-software delivery by non-technical users with and without LLM assistance, including review, debugging, and quality.
We found 42 claims in this post. See them all.
Full record
Source and dates
Where's Your Ed At, AI's Brokenomics, paragraph 114.
- Said
- We read the source
- live page, full text, public
- Evidence up to
- Checked
We read this wording on 2026-10-28. The post may have been edited after 2026-06-15.
Where it was said (1)
Said once in the posts we read.
: Where's Your Ed At, AI's Brokenomics
They’re not replacing anyone, and there isn’t compelling evidence that AI models speed people up. Allowing non-technical people to use LLMs to write code isn’t speeding up the delivery of software in a measurable way
- Where
- paragraph 114
- Published
- 2026-06-15
- We read it
- , live page
- Access
- full text, public
History
Versions
- Current: Contradicted. Evidence up to 2026-10-28.
What happened to this record
PublishedBy us
Published after both AI reviews and the final check passed.
Held backBy us
A reviewer said our test wasn’t fair to what was actually said.